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ABSTRACT 

These  proceedings  present  74  selected  abstracts  and 
47  selected  formal  papers  under  14  special  interest  group  headings. 
Topics  addressed  by  the  papers  include  constructing  multimedia; 
interactive  video;  computers  in  secondary  school  mathematics;  access 
in  computer-based  instruction;  implementing  computer-based 
technology;  advisor  development;  cognitive  maps;  software  development 
by  faculty-student  teams;   learning  environments  for  interactive 
technologies;  computer  assisted  design;  presentation  techniques; 
integrating  controversy  skills  into  computer  conferencing;  updating 
hypertext  systems;  navigation  maps  and  cognitive  styles;  utilizing 
hypermedia;  computer  supported  learning  in  administrative 
organizations;  creating  courseware  catalogs;  an  international 
multimedia  resource  center;  programming  paradigms;  using  the 
authoring  system  Quest;  developing  a  multipurpose  multimedia  lab; 
computer  graphics;  courseware  evaluation;  computer-based  tools  for 
methodology  teaching;  emerging  technologies;  interactive  multimedia 
courseware;  an  intelligent  framework  for  computer-based  instruction; 
patient  simulations;   interactive  audio;  managing  multimedia 
resources;  Digital  Video  Interactive;  metacogni tive  skills;  the 
instructional  design  taxonomy;  users'  mental  models;  building  expert 
system  rule  bases;   creating  an  intelligent  authoring/instructional 
system;   intelligent  tutoring  systems;  a  philosophy  of  education; 
computer-mediated  small-group  discussion;  computer-mediated 
conferencing;  and  networks  and  multimedia  systems.  Many  of  the  papers 
contain  references.  (ALF) 

Vc   Vc  ?  V  Vc  A    y  c   -k  '>\  Vc  Vc  ?'c  A  :V  Vc  A  V  r  ic  '>\   A  Vc  -kici^ik  Vc  Vc  V  c   A  -k      ic      Vc  Vc  *  V  c  Vc  Vc  Vc  Ve      A  Vc  A  yr  :^  A  A  A  ?V  Vc   Vc  A  A  Ve  Vc  Vc  A  i(  i(  Vc 

Reproductions  supplied  by  EDRS  are  the  best  that  can  be  made  * 
*  from  the  original  document,  * 

•kick-k  Ve i<  Ve -k  Ve   it  i<  Vc Vc  Vc  V e  Vc  Vc  -k   -k i<   Ve  Vc  Vc ic  ii  Vc  Vc   k      i<  Ve  V c  "k -k  i<  i< ic'k-k'ii-k-kic-k'kii      Vc  •kicici<-ki^i^i(i(i<'k  i<  Vc  A  A  Vc 


ERIC 


CO 


in 

CO 


Empowering  People 


U  S  DEPARTMENT  or  EOUCATtOW 

OM'ce  oJ  tducahonaJ  Resejfch  and  »/PO'ovement 

EDUCAUONAL  RESOURCES  INFORMATION 
CENTER  •ERIC' 

received  '"^"^  me  oe'so"  o'  o'Qa'><«di'C 
or<ginai«ny  .| 
C.  Mmo'  cnanges  nave  w*"'  nr^aae  ir  n-D'O^e 

reorooucnori  Qua'<l> 

#  PoinliO'  vie*  c  opinions  staled  ''  "'■s  dcx  .< 
ment  00  no»  necei!>a"<v  'eD'ese^;  o'lica 
06  R»  OOS't.on  O'  DOi«t» 


«5     Through  Technology 

ift  C7  Oft/ 


Conference  Proceedings  of  the 
34th  International  Conference  of  the 

Association  for  the  Development  of 
Computer-Based  Instructional  Systems 


ERIC 


ADCIS 


"PERMISSION  TO  REPRODUCE  THIS 
MATERIAL  HAS  BEEN  GRANTED  BY 

'J  ♦      »   i  1  r  3 1 1 LU  f  1 1 


TO  THE  EDUCATIONAL  RESOURCES 
INFORMATION  CENTER  (ERICI 

Norfolk  Waterside  Marriot  and  Convention  Center 
S  Norfolk,  VA 

November  8-11,  1992 


BEST  COPY  AVAILABLE 


Proceedings  of  the 

34th  Annual 
Internationa!  Conference  of  the 

Association  for  the 
Development  of  Computer-Based 
Instructional  Systems 

Norfolk  Waterside  Marriott  Hotel 
Norfolk,  VA 
November  8-11,  1992 

ADCIS  International  Headquarters 
1601  West  Fifth  Avenue 

Suite  1 1 1 
Columbus,  Ohio  43212 

Edited  by  Howard  Troutner 
Genesis  Management 

No  portion  of  this  document  may  be  reproduced  in  any  form  without  the  express  wri'ten  consent 
of  the  Board  of  Directors  of  the  Association  for  the  Development  of  Computer-based  Instructional 
Systems  or  their  duly  appointed  designee 


Table  of  Contents 


ELSECJC 

Special  Interest  Group  for  Elementary,  Secondary,  Junior  College 
Selected  Abstract^  for  ELSECJC 


Three  C AI  Programs  to  Teach  Chinese  to  Primary  School  Studenti  in  Taiwan,  Republic  of  China 

Yu  Kao,  Taiwan  Provincial  Irisntute  for  Elementary  School  Teachers  Inservice  Education  ? 

Quality  Courseware  for  Facilitating  Teachers'  Computer  Skills 

Doris  Lee,  Penn  State  Great  Valley   3 

PCpattem  as  a  Vehicle  for  Providing  Students  with  CAD  Experience 

Isabelle  M,  Lott  and  George  E.  Lott,  Pattern  Works  International   4 

Using  Hypermedea  and  Computer-Based  Instruction  with  Elementary  Children 

MarkL,  Sikorski,  Ktiapp  Elementary  School;  R.  Scott  Grabinger,  University  of  Colorado   4 

High  School  Students  Using  Supercomputers:  New  Horizons 

Mary  S,  Trainor  and  Don  Willerton,  Los  Alamos  National  Laboratory   5 

Using  the  PHOENIX  CBT  System  in  a  College  Learning  Environment 

Gary  Vaughn,  University  of  Cincinnati   J 

Selected  Formal  Papers  for  ELSECJC 
Constructing  Multimedia:  Solutions  for  Education 

Palmer  Agnew,  Ame  Kellerman,  andJeanine  Meyer,  IBM  Corporation   9 

A  Programmatic  Approach  to  Research  in  the  Application  of  Interactive  Video  to  Social  Learning 

Alan  Bain,  Brewster  Academy   17 

The  Process  of  Implementing  the  Use  of  Computers  in  Teaching  Secondary  School  Mathematics  and 
the  Effect  on  Student  Learning:  A  Field  Study 

M,  Peter  Jurhat,  Richard  Skov,  Edward  A,  Friedman,  Roger  S,  Pinkham  atid  John  McGinley,  S 

tevens  Institute  of  Technology  and  I^ri  Morris,  Educational  Testing  Service  33 

The  Effects  of  Questioning  Access  in  Computer-Based  Instruction 

Joan  C.  Lee  and  Richard  Dennis,  University  of  Illinois  at  Urbana-Champaign  53 

Interdisciplinary  Multunedia  Learning  Using  Anchored  Instruction 

Robert  J,  Myers,  Virginia  Polytechnic  Institute  and  State  University   73 


Make  it  Happen:  Implementing  Computer-Based  Technology  in  School  Systems 

Kristie  PosterOt  University  of  Georgia  *  81 

ERIC 


ETSIG 

Special  Interest  Group  for  Emerging  Technolo^es 


Sglgcted  Afrstracfe  fQr  etsig 

Designing  Computer-Supported  Intentional  I^msng  Environments 

Joawuz  C.  Dunlap  atid  R,  Scott  Grahinger,  University  of  Colorado  97 

Electronic  Performance  Support  Systems  Demonstrated  to  Reduce  Cholesterol  Among  Users! 

Robert  C.  Fratini,  AT&T Natiotial  Product  Training  Center   97 

Evaluating  Mindtools  in  Schools 

David  H»  Jonassen,  University  of  Colorado  98 

Exploration  and  Experience  with  Second  Generation  Authormg 

Paul  J.  Magelli,  Software  To  Educate  People,  Inc  95 

Building  Sophisticated  Learning  Software  in  Multimedia 

John  E.  Whitaker,  Arthur  Anderson  &  Co,  Societe  Coopertive  98 

A  Computer-Based  Simulation  of  Planned  Change 

Michael  Yacci,  Rochester  Institute  of  Technology  99 

Selected  Formal  Papers  for  ETSIG 
Advisor  Development  Life  Cycle 

Marie-Michele  Boulet,  Universite'  Laval,  Quebec,  Canada  103 

Cognitive  Maps  Before  and  After  Using  a  Knowledge  Construction  Er  ironment 

Joanna  C  Dunlap  and  R.  Scott  Grahinger,  University  of  Colorado  Ill 


HESIG 

Special  Interest  Group  for  Health  and  Medical  Education 
Selected  Abstracts  for  HESIG 

Using  Multimedia  to  Provide  Information  About  Childhood  Cancer  to  Elementary  Grades 


IdaM.  Flynn,  University  of  Pittsburgh  123 

Computer-Assisted  Learning  -  Tools  vs.  Appliances 

John  A.  Mack,  XR  Data  Systems  123 

Student  Access  to  Hospital  Information  Systems:  Problems  and  Approaches 

Cynthia  M»  Mascara  ami  Ramona  Nelson,  University  of  Pittsburgh  Medical  Center  124 

Clinical  Nursing  Experience  in  Labor  and  Delivery  Via  Interactive  Video 

Dee  McGonigle,  Pennsylvania  State  University  124 

5 

O  '  ii 

ERIC 


Selected  Formal  Papers  for  HESIG 
Faculty-Student  Teams  Develop  CBL  Software 

Anthony  J »  Frisby  and  Rodney  B.  Murray,  Uxonias  Jefferson  University  127 

Designing  EfTective  and  Safe  Learning  Environments  for  Interactive  Technologies  -  Guideline's  for 
the  Health  Professions 

Michael  Weisherg,  National  Institutes  of  Health,  National  Library  of  Medicine  133 


HOMEC 

Special  Interest  Group  for  Home  Economics  Education 
Selected  Abstracts  for  HOMEC 

Development  of  an  Interactive  Multimedia  System  Presenting  Food  Safety  Education  to  Nav^o  Audiences 


Jeanne  Gleason,  New  Mexico  State  University  155 

Selected  Formal  Papers  for  HOMEC 

CAD:  An  Expressive  Partner,  A  Challenge,  A  Violin! 

Stephanie  A.  Clemons,  Colorado  State  University;  Lisa  Waxnian,  Florida  State  University 

and  Diane  Davis,  Valparaiso  University  159 

Effects  of  Presentation  Techniques  on  CAD  Workshop  Participants'  Attitudes 

Stephanie  A.  Cleniofis  atid  Peggy  S,  Berger,  Colorado  State  University  165 

Integrating  Controversy  Skills  into  Computer  Conferencing:  Parameters  for  Instructional  Model  Building 

James  L.  Morrison,  University  of  Delaware  and  Pamela  P.  Morrison,  Goldey-Beacom  College  177 


HYPERSIG 

Special  Interest  Group  for  HyperMedia 


Selected  Abstracts  for  HYPERSIG 

Teaching  with  HyperCard:  An  Innovative  Case  Study 

BryoJiR.  Cole,  Texas  A  iScM  University  185 

Simulating  the  Reasoning  Process  with  Causal  Modeling  Techniques 

Joanna  C.  Dunlap  and  David  H.  Jonassen,  University  of  Colorado  185 

An  Examination  of  Novice  Linking  in  Hypermedia  Environments 

Steve  \y,  Harmon,  University  of  Houston  -  Clear  Lake;  Tliomas  C.  Reeves,  Tlie  University  of  Georgia  186 

The  Media  Mix 

Steve  Hannon,  University  of  Houston  -  Clear  Lake.  136 

Cognitive  Flexibility,  Epistemology,  and  Hypertext  Learning  Environments:  Research  into  the 
Transfer  of  Complex  Knovirledge 

Michael  J.  Jacobson,  University  of  Illinois  * .  * . »  * ...» ,  »  7  57 


in 


ERIC 


Conceptual  and  Methodological  Problems  in  Hypermedia  Research 

David  H»  Jonassen,  University  of  Colorado  187 

Coliaborative  Knowledge  Construction  in  an  Argument-Based  Hypertext 

David  H.  Jonassm,  University  of  Colorado  188 

The  Application  of  Research-Based  Instructional  Design  Principles  in  Developing  a 
Hypermedia-'Assisted-Instnictional  Courseware  for  Second  Language  Learning 

MinLiu,  West  Virginia  University  ^  188 

Information  Seeking  -  A  Fundamental  Learning  Activity 

Carol  B*  MacKnight,  University  of  Massachusetts  188 

A  Course  Design  for  Going  Beyond  HypeiTalk 

Rose  M,  Marra,  AT&T  Bell  Labs;  R.  Scott  Grabinger,  University  of  Colorado  189 

Wildlife  Wonders:  Validating  a  Prototype  for  Multklisciplenary  Education  for  Middle  School 

Gayle  Af.  Munsofi  and  Eric  E.  Smith,  University  of  Northern  Colorado  190 

Hypermedia  Development  Issues:  Validated  Guidelines  and  Research  Needs 

Gayle  Af.  Munson,  Tom  Lightner,  and  Eric  E.  Smith,  University  of  Northern  Colorado  190 

Effectiveness  of  Interactive  Video  on  Tennis  Skill  Analysis 

Gayle  M.  Munson  and  Eric  E,  Smith,  University  of  Northern  Colorado  191 

Evaluatmg  Hypermedia  Environments:  Insights  From  the  Perstus  Project 

Delia  Neuman,  University  of  Maryland.  191 

Building  Blocks  -  A  Literacy  Program 

Laurie  Rosenthal,  Teachers  College/  Columbia  University  192 

Developmental  Formative  Research  on  Interactive  Materials 

Erik  Strommen,  Children's  Television  Workshop;  Barbara  Flagg,  Multimedia  Reseqrch; 

Patricia  R,  Garretson,  IBM  Corporation;  atid  Henry  Olds,  Bolt,  Beranek  <fe  Newman  192 

Selected  Formal  Papers  for  HYPERSIG 
A  Strategy  for  Updating  Large  Hypertext  Systems 

John  M,  Jackson,  University  of  Virginia  Medical  School  195 

Navigation  Maps  and  Cognitive  Styles  in  Hypertext  Assisted  Learning 

Paul  M.  Leidig,  Grand  Valley  State  University.  203 

Strategies  for  Utilizing  Hypermedia:  Instruction  v.^.  Information  Access 

Xiaoming  Zhu,  Electronic  Data  Systems  Corporation  223 


7 


IV 


MISIG 

Special  Interest  Group  for  Management  Issues 
Selected  Abstracts  for  MlSlG 

Considering  Organizational  Culture  in  Selecting  CBT  Development  Software 

Robert  C.  FraCini,  AT&T Natiotuxl  Product  Training  Cemer,  235 

Lifelines  for  the  Struggling  Multimedia  Developer:  Living  os  Multimedia^s  Bleeding  Edge 

Jeanne  Gleason,  New  Mexico  State  University  235 

Selected  Forma!  Papers  for  MISIG 

Quality  Improvement  Cycle  aikd  Con^^uter  Supported  Learning  in  Administrative  Organizations: 
An  Observation  Through  Some  Experiments 

Gianpiero  Bussolin,  Universito  di  Torino  239 

MUSIC 

Special  Interest  Group  for  Music 
Selected  Abstracts  for  MUSIC 
Object  Oriented  Music  Theory  Courseware 

Paul  E.  Dworak,  University  of  North  Texas.  247 

PUG 

Special  Interest  Group  for  Plato  Users 
Selected  Formal  Papers  for  PUG 

Creating  Courseware  Catalogs:  The  Marriage  of  Database  Technology  to  CBI  Systems  - 
An  Insider's  View 

Elizabeth  C.  Miller  atui  Pamela  T  Hess,  Control  Data  Corporation  251 

SIGAC 

^Special  Interest  Group  for  Academic  Computing 
Selected  Abstracts  for  SIGAC 

Examples  of  Instructional  Empowerment  Using  ToolBook 

Carol  Dwyer  and  Morris  Weinstock,  Penn  State  University  261 

The  Student  of  Socrates  Meets  the  Students  of  Kentucky  PLATO(R)  and  Western  Kentucky  University 

Kenneth  L.  Modesitt,  Western  Kentucky  University  ,  261 


Reality-Based  Strategic  Planning  for  Computing  in  Universi^^-BfiSed  Schools/Departments 

Ramom  Nelson,  University  of  Pittsburgh  Medical  Center  arid  Pamela  liepple, 

University  of  Pittsburgh  School  of  Nursing  262 

Faculty  Development  and  Support  for  Development  and  Utilization  of  Computer-Based  Instruction: 
Experiences  along  the  Continuum 

Floyd  K.  (Rusty)  Russell,  Stan  Cohen,  andAmie  Nardi,  West  Virginia  University;  Carol  Dwyer, 

Pennsylvania  State  University  mid  Charlotte  Scherer,  BovAing  Green  State  University  263 

Communicating  Knowledge  about  LANs 

Donald  Scherer,  Bowling  Green  State  University  ^^-...265 

Multimedia  Lectures:  A  Comparison  of  Some  Macintosh  Alternatives 

Eileen  E,  Schroeder,  Pennsylvania  State  University  264 

Selected  Formal  Papers  for  SIGAC 

The  Design,  Implementation^  and  Management  of  Lehigh  University's  Internationa!  Multimedia 
Resource  Center 

Blair  R,  Bemhanlt,  Lehigh  University  267 

A  Computer-Based  Course  in  Programming  Paradigms 

Allen  N  Brown,  Queen's  University  275 

Using  the  Authoring  System  Quest  to  Implement  Courseware  in  Computer  Science 

Tomasz  Muldnerand  Earle  Morven  Lowe,  Jodrey  School  of  Computer  Science  289 

Making  the  Low  Bid  Work  in  Developing  a  Multipurpose  Multimedia  Lab 

Eric  E.  Smith,  Guy  W.  Wesihqff,  Tom  Lightner,  and  Rich  Brown,  University  of  Northern  Colorado  301 


SIGCBT 

Special  Interest  Group  for  Computer-Based  Training 
Selected  Abstracts  for  SIGCBT 

The  Use  of  Computer-Based  Instructional  Simulations  for  Teaching  Photography  Skills 


Charles  F.  Belinski,  Tlje  University  of  Georgia  311 

Designing  Computer-Based  Assessment 

James  H,  Cowardin,  Precision  Learning  Systems  ,  311 

Are  Software  Quality  Factors  a  Relevant  Measure  of  CBT  Quality? 

Michael  J.  Hillelsohsu  McDonnell  Douglas  Space  Systetns  Company  312 

Effective  Screen  Design  for  Computer-Based  Training  and  Interactive  Video 

William  D,  Miltieim,  Perm  State  Great  Valley;  Bretida  Batman  Haag,  Penn  state  Utiiversity  312 

Quality  CBT  -  The  Quick  Way 

Jeff  Oliver t  CastU  Learning  Systems  Ltd   313 

9  vi 


Credsntialing  Courseware  Developers:  A  Postgraduate  Program  in  Computer  Basec  Learning 

Rod  Sims,  University  of  Technology,  Sydney  


313 


Meta-CBT  2:  The  Final  Product 

Ro{iSims,  University  of  Technology,  Sydney  314 

A  HyperCard/Hypermedia  Courseware  Package  Entitled:  "Ticked  Off!  -  A  Resource  for  Tick 
Identincation" 

Wendy  Snetsinger,  Pemx  State  University  314 

The  Ford  Experience 

Titnothy  W.  Spannaus  attd  Laurie  Diener,  Vie  Enidicium  Group  Inc;  Jackie  Binkert,  Ford  Motor  Co  J/5 

Scoping  A  Program  for  Delivering  CBT  to  Your  Users:  A  Hands-Ou  Planning  Session 

Adele  Suchiftsky,  US  Gefieral  Accounting  Office  315 

Selected  gormai  Papers  for  CBT 

Computer  Graphics:  Implications  for  Instructional  Design 

Lih'Juan  ChanLin,  James  R,  Okey,  atid  David  Re  inking,  University  of  Georgia  319 

Courseware  Evaluation:  Techniques  from  Instructional  Design  Theory  and  Cognitive  Science 

Dale  y.  Hoskisson,  Valley  City  State  University  335 

Computer-Based  Tools  for  Methodology  Teaching 

Achilles  Kameas,  P.  Pintelas,  and      Crampes.  University  of  Patras  341 

in  Search  of  New  Ideas,  Research  Findings,  and  Emerging  Technologies?  Here's  Where  to 
Find  Them 

Gary  C,  Pov^ell,  University  of  Georgia  357 

Interactive  Multimedia  Courseware;  A  Perspective  on  Competency-Based  Training  for  the  Hospitality 
Industry 

RodSinis,  University  of  Technology,  Sydney  363 

An  Intelligent  Framework  for  the  Creation  of  Effective  Computer-Based  Instruction 

J.  Michael  Spector,  and  Daniel  J.  Muraida,  Armstrong  Laboratory,  Tech  Training  Research  Division  373 

CBT  -  The  Soft  Way 

Loma  Uden,  Staffordshire  Polytechnic  School  of  Computing  381 

SIGIVA 

Special  Interest  Group  for  Interactive  Video/ Audio 
Selected  Abstracts  for  SlGlVA 

Development  of  m  Intcniuive  Multimedia  System  for  Use  in  a  Public  Access  Environment 

Jeanne  Gleason,  New  Mexico  State  University  403 

ERIC  ^"^^ 


A  MICAL  3Projcct  Update 

Darleen  V.  Figford  afxd  Greg  Baur,  Western  Kentucky  University, 


403 


An  Interactive  Video  Deiesonstratlon  Tutorial  of  the  Visual  Almanac  Empowers  Preservice  and 
Inservice  Teachers 

Oiarlotte  Scherer,  Bowling  Green  State  University  404 

Knowledge  and  Skills  of  Team  Members  for  the  Preproduction  Stage  of  Developing  IVD  Courseware 

Carol  Smith,  West  Virginia  University  404 

Empowerment  Through  Museums  and  Multimedia 

Paul  Wangemann,  Motorola  Museum  of  Electronics  405 

Embedding  Instructional  9>elivery  Systems  for  Maximum  Effectiveness 

Stuart  H.  Weinstein  ami  William  Skipper,  SWL,  Inc  405 

Selected  Formal  Papers  for  SIGIVA 

Adaptive  Instructional  Use  of  Interactive  Videodisc  Patient  Simulations 

Paunpimon  Amomkul,  Lowell  A.  Sckoer,  and  Michael  W.  Finkelsiein,  University  of  Iowa  409 

/nteractive  Audio:  Research  and  Practice 

Ann  Barron,  University  of  Southern  Florida  419 

Partners  in  Interactive  Video:  Designing  for  Research  and  Implementation 

Joseph  G,  Gregg,  Douglas  Morton,  Eric  E,  Smith,  D.  Allen  Phillips,  University  of  Northern  Colorado  425 

Using  a  ToolBook  Database  Front  End  to  Manage  Multimedia  Resources 

C.  Michael  Hassett  atui  Mary  R.  Hassett,  Fort  Hays  State  University  433 


A  Primer  on  DVI ;  Digital  Video  Interactive 

Darleen  V.  Pigford,  Western  Kentucky  University  443 


SIGPILOT 

Special  Interest  Group  for  Users  of  Pilot 


Selected  Abstracts  for  PILOT 


The  Comp  '<bility  Problem  in  Designing  and  Developing  Educational  Software  With  the 
New  Gene    jon  of  Smart  Tools 

Emile  Edward  Attala  and  Tommy  DeMoville,  Cat  Poly  State  University  


.45  J 


II 

viii 


SIGTAR 

Special  Interest  Group  for  Theory  and  Research 


ERIC 


Selected  Abstracts  for  SIGTAR 

The  Application  of  Motivational  Principles  to  Visual  Screen  Design 

Charles  F,  Belitxski  and  Jimmy  Williamson,  Vie  University  of  Georgia  455 

Computer  Adaptive  Testing:  Preparation  and  Analysis  of  the  Item  Pool 

Randal  D,  Carlson,  Pennsylvania  State  University » . . ,  455 

Models  for  Using  Computer  Facilitated  Communications  to  Enhance  Instructional  Interactivity 

Norman  R,  DodU  D,  Michael  Moore,  atxd  John  K.  Burton  t  Virginia  Tech  456 

What  is  the  Place  of  Theory  in  Instructional  Design  and  Educational  Technology?  A  Discussion  of  the 
Implications  for  Research  and  Application 

Michael  J.  Jacobson,  University  of  Illinois,  Steven  Tripp,  University  of  Kafisas;  M,  DavidMerrill, 

Utah  State  University;  David  Jofiassen,  University  of  Colorado;  Cynthia  Leshin,  Maricopa  Community 

College;  and  Barbara  Grabowski,  Tfie  Pennsylvania  State  University  456 

Automating  Instructional  Design  for  Teaching  Dynamic  Processes 

Mark  K,  Jones,  Utah  State  University  457 

Integrated  Instruction  for  Complex  Computerized  Tools 

J  {/an  C.  Lee,  US  Army  Construction  Engineering  Research  Lab  457 

Accounting  Instruction  through  Structured  Learning  and  Elaboration  Theories  : 
A  Hypertext  Comparison 

Tom  Lightneraml  Eric  E.  Smith,  University  of  Northern  Colorado  458 

Can  Graphical  Browsers  Affect  a  Learner's  Structural  Knowledge  in  a  Hypertext  Environment? 

Eileen  Schroeder,  Pennsylvania  State  University  453 

The  Instructional  Media  Research  Debate:  Medium  or  Message? 

Vieodore  M.  Shlechter,  Army  Research  Institute  and  J.  Dexter  Fletcher,  Institute  for  Defense  Analysis  459 

The  Application  of  Cognitive  Flexibility  and  Constuctivisni  Theories  to  a  Hypermedia  Course 
on  Thermal  Processing 

Wendy  Snetsinger  and  Susan  Clark,  Pern  State  University  459 

The  Effects  of  Audio  Resources  on  Learning  form  a  Hypertext  Bilingual  Dictionary 

Steven  Tripp,  University  of  Kafisas  t  460 

The  Effects  of  Graphical  Display  Characteristics  in  a  Computer-Base/d  Environment 

James  B.  We  lis  ^  North  Georgia  College;  Ben  H.  Layne,  Georgia  State  University  460 

Computer-Based  Research  Formats:  Utilizing  HyperCard  to  Administer  Research  Projects 
and  Collect  Data 

Ronald  Zeliner,  Texas  A<kM  University  461 


Selected  Forma!  Papers  for  SIGTAR 


Metacognitive  Skills  in  Computer-Assisted  Instruction 

Myung-Sun  Clinton  a  fid  James  /?.  Okey,  Tlie  University  of  Georgia  465 

The  Instructional  Design  Taxonomy:  Further  k^;lnemcnts  for  Describing  and  Measuring 
the  Interactivity  of  Computer-based  Instruction 

Dale  y.  Hoskisson,  Valley  City  State  University  473 

The  Structure  of  Interfaces  and  Users'  Mental  Models  in  Computer-Based  Interactive  Courseware 

Hueyching  Janice  Jih,  Institute  for  Information  bidiistry    479 

Effects  of  Building  Expert  System  Rule  Bases  on  Cognitive  Structures  and  Critical  Thinking 

David  H.  Jonassen,  University  of  C:^lorado  499 

A  Methodology  for  Creating  an  Intelligent  Authoring/Instructional  System 

Janet  T.  Kerne r,  St.  John's  University...,  509 

Does  ITS  Help»  Help?  and,  Is  an  ITS  Error  Diagnosis  a  Remedy? 

Michael  Orey,  Joo-siing  Park,  Lih-juan  Clianlin,  Phil  Gillis,  ofui  Peter  Legree,  7he  University  of  Georgia  5/7 

Research  an  i  CBi  in  Business:  The  Need  for  Dialogue 

Gary  C.  Powell  and  TJiomas  C  Reeves,  University  of  Georgia  531 

What  LS  Has  Learned 

Steven  Tripp,  University  of  Kafisas  547 

Michael  Oakeshott's  Philosophy  of  Education  and  its  Implications  for  Instructional  Design  Theory  and 
Practice 

Steven  Tnpp,  University  of  Kafisas  559 


SIGTELE 

Special  Interest  Group  for  Distance  Learning  and  Telecommunications 
Selected  Abstracts  for  SIGTELE 


Inter-High  School  Writing  Project 

Darkne  Chirolas,  University  of  Illinois  579 

Using  ERIC  -  A  Database  and  More 

JaneJaflis,  ERIC  Clearifighouse  on  Ififormation  Resources  579 

579 

Selected  Formal  Papers  for  SIGTELE 


Effects  of  Anonymity  and  Group  Saliency  on  Participation  and  Interaction  in  a  Computer-  mediated 
Small-i^roup  Discussion 

Trrfficf  Ahrnt  atui  Vance  Durrifigton,  Texas  Tech  Ufiiversity.  »  583 


Enhancing  Context  and  Continuity  in  Computer-mediated  Conferencing:  Developing  the  IdeaWeb 

Terence  Ahem,  Texas  Tech  University,  589 

Networks  and  Multimedia  Systems 

Greg  Ban r.  Western  Kentucky  University  60] 

Computer  Mediated  Communications:  How  Does  it  Change  the  Social-Psychological  Aspects  of 
Teaching  and  Instruction? 

Laurie  F.  Ruberg  and  Vionuxs  M.  Sherman,  Virginia  Tech  609 


i  ^ 

xi 


Selected  Abstracts  for  the 

Special  Interest  Group  for 
Elementary,  Secondary,  and 
Junior  College  (ELSECJC) 


ERIC 


IT 


Three  CAi  Program  to  Teach  Chinese  to  Prunary  School  Students 
in  Taiwan^  Republic  of  China 

YuKao 


The  ideals  of  "Teaching  without  discrimination"  and  "Teaching  according  to  talents"  have  always 
been  the  core  of  the  educational  system  in  Taiwan.  Computers,  with  the  characteristics  of 
individualization  and  interaction,  can  fulfill  the  demands  to  a  satisfactory  degree.  The  Ministry 
of  Education  in  this  country  has  acknowledged  the  international  trend  to  use  computers  in 
education  and  has  made  plans  to  promote  computer-assisted  instruction  (CAI)  at  all  levels  of 
school  in  Taiwan. 

The  purpose  of  this  research  is  to  develop  CAI  software  for  primary  school  students.  In  order 
to  help  the  children  in  Taiwan  learn  through  computers  in  the  easiest  way,  the  Information 
Science  &  Technology  Center  has  developed  three  CAI  programs  to  teach  primary  school 
students  Chinese. 

These  three  programs  are  Chinese  Phonetic  Symbols,  Chinese  Four-Word  Phrases,  and  Tang 
Poetiy.  These  programs  enable  the  students  to  learn  both  the  basic  knowledge  of  the  Chinese 
language  and  the  operation  of  the  computer.  Moreover,  English  directions  are  available. 
Students  can  learn  in  a  bilingual  environment.  Afterwards,  four  approaches  including  analysis, 
design,  implementation  and  evaluation  are  adopted  to  develop  a  game  model  of  CAI  software 
for  future  reference. 

Finally,  conclusions  and  implications  of  this  research  are  discussed.  Further  studies  are 
suggested. 


Quality  Courseware  for  Facilitating  Teachers'  Computer  Skills 

Doris  Lee,  Penn  State  Great  Valley 

Many  educators  have  revealed  that  often  school  teachers  are  not  will  trained  to  use  computers 
due  to  inadequate  training  methods.  Common  problems  found  in  training  software  are 
inadequate  context  and  help,  one  type  of  instructional  lesson  for  diverse  learners,  insufficient 
practice  opportunities  for  novice  users  and  inappropriate  responses  to  learners'  learning 
difficulties. 

Quality  courseware  that  is  the  systematic  design  of  computer  software  :>hould  be  used  for 
facilitating  school  teachers'  computer  skills.  These  skills  may  include  operating  hypermedia, 
desktop  applications,  instructional  courseware,  or  acquiring  simple  hardware  and  programming 
techniques.  Effective  courseware  can  be  designed  to  have  a  comparison  between  the  new 
technology  and  other  mechanisms  with  which  the  learner  is  more  familiar.  It  should  offer  a 
higher  level  of  learner  control  in  which  learners  can  plan  their  own  learning  activities.  Finally, 
quality  courseware  should  provide  learners  with  more  query  and  responding  opportunities  for 
encouraging  self-correction. 
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PCpattern  as  a  Vehicle  for  Providing  Students  with  CAD  Experience 

Isabelle  M.  Lou  and  George  E,  Lott,  Pattern  Works  International 

The  challenge  to  teachers  in  the  1990's  is  to  see  that  educational  vehicles  are  available  which 
will  provide  all  students  with  a  means  to  acquire  requisite  computer  literacy. 

Many  secondary  schools  and  junior  colleges  have  installed  computer  laboratories  and  computer- 
aided  design  software.  However,  the  majority  of  the  use  of  these  laboratories  is  by  individuals 
interested  in  construction,  engineering,  and  the  design  of  manufacturing  equipment  and  parts. 
Students  with  other  career  goals  are  not  acquiring  CAD  skills. 

PCpattern,  an  AutoCAD  add-on  garment  design  software  product,  is  the  ideal  vehicle  to  use  to 
open  the  world  of  computer-aided  design  to  this  untapped  student  population.  The  flexibility  and 
low  cost  of  PCpattern  means  that  teachers  can  quickly  and  easily  capitalize  on  the  inherent 
student  interest  in  clothing.  The  potentials  of  the  software  can  be  used  to  design  relatively 
simple  garment  patterns  in  a  short  space  of  time,  thereby  exposing  students  to  AutoCAD. 

Students  who  have  a  more  intensive  interest  can  continue  to  learn  the  process  of  garment  design. 
The  software  can  be  used  for  an  entire  course  in  garment  design  using  CAD,  or  it  can  be  quickly 
and  easily  adapted  to  the  independent  study  approach  using  our  educational  materials  and 
tutorials. 


Using  Hypermedia  and  Computer-Based  Instruction  with  Elementary  Children 

Mark  Sikorski,  Knapo  Elementary  School;  /?.  Scott  Grabinger,  University  of  Colorado 

HyperCard  and  HyperTalk  are  often  used  to  create  lessons  for  use  with  middle  and  high  school  students. 
Little  documentation  exists  on  their  use  with  young  elementary  age  learners.  The  purpose  of  this  session 
is  to  report  on  two  teacher-created  activities  that  meet  specific  needs  of  first  through  fifth  grade  students 
in  mathematics  and  reading/writing. 

The  relevancy  of  interactive  technology  like  hypermedia  and  computer-based  instruction  is  a  key  to 
maximizing  learner  interest  and  choice  in  the  learning  environment.  This  presentation  demonstrates  a 
program  used  to  teach  second  grade  students  about  place  value.  Anecdotal  data  will  be  presented  to 
describe  the  effect  on  these  young  students.  Data  from  another  program  used  with  fifth  graders  in 
reading  and  writing  activities  will  also  be  presented. 

The  session  will  be  presented  in  a  lecture  format  using  handouts  and  overhead  transparencies  (see 
appendix  for  samples).  Content  of  the  session  includes: 

I.  Introduction 

-description  of  learning  environments,  children  and  programs 

II.  Needs  Analysis  Results  and  Objectives  Development 

III.  Program  Design  Decisions 

IV.  Program  Creation 

V.  Results  of  Program  Use 

-test  results 
-anecdotal  data 

VI.  Conclusions  and  Future  Possibilities 


High  School  Students  Using  Supercomputers:  New  Horizons 

Mary  Trainor  and  Don  Willerton,  Los  Alamos  National  Lab 

As  computers  are  used  in  almost  every  workplace  today,  it  is  important  that  high  school  students 
obtain  exposure  to  computers.  That  exposure  normally  ranges  from  use  of  a  word  processing 
to  programming.  Microcomputers  can,  however,  handle  only  a  small  fraction  of  the  nation's 
problems.  Supercomputers  are  heavily  used  to  solve  major  research  problems  of  national 
interest.  These  problems  vai-y  from  analysis  of  new  automobile  engine  designs  to  modelling  of 
the  earth's  atmosphere  to  see  the  effects  of  different  pollutants.  It  is  important  therefore  for 
students  to  understand  and  experience  the  range  of  compute  power. 

Los  Alamos  National  Laboratory  is  now  in  its  third  year  of  supporting  the  New  Mexico 
Supercomputing  Challenge.  This  academic-year  long  program  open  to  any  high  school  student 
in  the  state  provides  student  teams  the  opportunity  to  devise  a  research  problem,  write  a  program 
to  solve  that  problem,  and  run  the  program  on  a  supercomputer.  The  teams  work  under  the 
supervision  of  their  teacher  and  with  the  help  of  a  professional  scientist  coach.  Support  for  the 
Challenge  is  complex,  in  light  of  the  varied  equipment  and  connection  needs  of  the  schools,  the 
varied  backgrounds  of  the  students  and  teachers,  and  the  range  of  problems  chosen  by  the  teams. 
This  support  is  provided  by  a  partnering  of  academic,  government,  and  industrial  sponsors  in 
the  state  of  New  Mexico.  Los  Alamos  provides  use  of  the  computers,  consulting  and  coaching 
support,  administration,  security,  and  education. 

The  Challenge  doubled  in  size  from  tlie  first  to  second  year*  In  1991-92  there  were  419  student 
participasits  on  112  teams  from  51  schools.  These  students  used  over  250  CPU  hours  .on  the 
Crays  at  Los  Alamos  this  year.  Each  year,  awards  are  given  to  many  of  the  teams.  The 
projects  of  the  teams  that  won  the  grand  prize  were:  "The  Critter:  An  Artificial  Life  Project" 
and  "The  Rings  of  Saturn:  A  Simulation."  This  success  of  the  Challenge  has  been  attributed  to 
the  nonselective  nature  of  the  Challenge,  the  teamwork  format,  and  the  degree  of  excitement  and 
opportunity  associated  with  using  a  supercomputer. 

The  panel  will  not  be  a  project  report,  but  rather  a  discussion  of  issues  associated  with  use  of 
computers  in  secondary  education. 

Using  the  PHOENIX  CBT  System  in  a  College  Learning  EnYironment 

Gary  Vaughn,  University  of  Cincinnati 

This  oral  presentation  will  discuss  the  issues  and  concenis  involving  the  usage  of  a  mainframe 
computer-based  training  system  in  a  two-year  college  environment.  The  Computer-Assisted 
Instruction  Lab  of  the  University  College  of  the  University  of  Cincinnati  provides  learning 
opportunities  for  students  enrolled  in  programs  offered  by  the  Language  Arts,  Mathematics, 
Business,  and  Humanities  departments.  The  lab  has  over  forty  mainframe  modules  available  to 
students  who  drop  in  on  their  own  initiative,  who  are  referred  by  their  instructors  individually 
or  as  a  class,  or  who  register  for  accredited  courses  held  solely  in  the  lab.  Software  ranges  from 
developmental  to  intermediate  skills  levels,  in  keeping  with  the  college's  mission  to  prepare 
students  for  transfer  into  the  various  baccalaureate  colleges.  The  lab  has  a  total  of  sixteen 
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terminals,  eight  IBM  dumb  terminals  and  eight  Zenith  236  PC's  which  are  used  for 
microcomputer  leamware  as  well  as  terminal  emulation.  The  PC*s  connect  to  the  University's 
Amdahl  mainframe  via  Procomm  communication  software,  a  new  digital  phone  system  that 
enables  transmission  at  9600  baud,  and  three  IBM  7171  Protocol  Converters  with  over  ninety 
ports.  The  IBM  terminal  emulation  also  allows  students  to  connect  with  UCLID,  the 
University's  on-line  library  catalog  system.  The  lab  is  staffed  by  three  to  four  students  proctors 
and  a  faculty  director  supervised  by  the  Assistant  Dean  for  Academic  Services  and  guided  by 
a  Computer  Advisory  Committee. 
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ABSTRACT: 

Our  objective  is  to  show  how  educators  can  help  students  to  achieve  desired  pedagogical  goals  by 
explicit  classification  and  dcfmition  of  objectives,  coaching  students  in  the  construction  of  their  own 
multimedia  essays,  and  encouraging  a  practice  of  reflection. »  Rather  than  hearing  details  about  new 
multimedia  hardware  and  software,  readers  and  session  attendees  will  learn  about  a  framework  of 
goals  and  a  process  for  teachers.  We  will  relate  results  of  our  experiences  in  using  these  techniques 
with  educators. 

THE  PROBLEM 

Many  educators  have  heard  that  using  new  multimedfa  technolo^es  helps  students  learn,  not  only 
by  motivating  the  students  to  spend  more  time  on  task,  but  also  by  taking  advantage  of  students' 
ability  to  interact  closely  with  diifferent  media  such  as  text,  audio,  graphics,  images,  animation,  and 
video.  A  great  deal  of  multimedia  hardware  and  software  products  are  available,  along  with  some 
information  about  how  to  make  them  run.  Much  less  information  is  available  about  how  to  use 
thenr.  to  achieve  particular  desired  student  outcomes.  A  New  York  City  Public  School  educator, 
Dr.  Florence  Mann,  responsible  for  the  Technical  Assistance  Centers'  staff  development  program, 
crystallized  this  problem  for  us  by  stating  that  teachers  needed  to  understand  "why'',  not  just  "how* 
to  use  multimedia  effectively  in  the  teacliing  and  learning  environment.  This  became  our  challenge. 


OUR  SOLUTION 

We  surveyed  many  multimedia  products  and  many  possible  ways  to  use  them  in  classrooms.  We 
concluded  that  one  of  the  most  promising  ways  to  use  them  was  for  teachers  to  coach  and  facilitate 
students  in  creating  their  own  multimedia  materials.  We  found  that  we  could  devise  projects  that 
would  help  students  meet  useful  pedagogical  goals  in  three  very  distinct  categories.  One  is  a  very 
broad  set  of  objectives,  wliich  we  term  higher  order  tliinking,  meta-cognilive,  and  human  dynamics 
skills.  A  second  category  includes  the  content  area  or  discipline  specific  concepts  and  facts  which 
would  be  described  in  a  curriculum.  Lastly,  we  have  the  category  of  technical  /  mechanical  skills. 
Here  are  examples  within  each  of  the  three  categories: 

Under  the  higher  order  category,  we  include  ability  to  use  complex  concepts;  knowledge  of  vocab- 
ulary, symbolism,  and  interpretation;  choices  of  ways  to  organize  information;  understanding  nav- 
igation and  tours  through  information;  ability  to  summarize  and  choose,  ability  to  present 
information  appropriately;  working  successfully  in  a  group;  and  so  on. 

The  content  are  includes  the  objectives  of  the  subject  area  addressed,  such  as  science,  math,  social 
studies,  literature,  etc..  These  would  be  the  significant  facts  and  concepts  on  specific  or  interdisci- 
plinary topics  and  also  any  use  of  media  to  exhibit  the  sense  of  an  era  or  drarna. 
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Constructing  multimedia  or  any  other  type  of  essay  involves  specific  technical  or  mechanical  skills. 
Examples  arc  authoring  of  multimedia  projects;  good  use  of  an  muhimcdia  authoring  tool's  capa- 
bilities; hypermedia  linking  to  organize  materials;  good  planning  of  interim  milestones;  appropriate 
use  of  audio  recording,  pictures,  and  video;  knowledge  of  some  technologies  used  in  multimedia 
systems. 

Of  course,  people  will  differ  on  this  type  of  taxonomy.  We  offer  this  as  a  model  and  strongly  urge 
teachers  to  develop  their  own. 

We  devised  and  refined  four  projects  that  could  be  used  not  only  to  introduce  successive  multimedia 
technical  skills  but  also  to  meet  higher  order  thinking  skill  goals  and  content  goals  in  four  lists  such 
as  the  one  above.  We  decided  to  teach  teachers  how  to  use  these  projects  by  giving  them  the  op- 
portunity to  spend  a  week  acting  as  if  they  were  their  own  students  and  actually  creating  the 
projects.  Dr.  Mann  agreed  to  provide  some  members  from  her  group  of  highly  competent  tech- 
nical support  personnel  and  a  class  of  interested  teachers,  so  that  we  could  see  if  our  combination 
of  ideas  worked.  This  paper  reports  that  it  was  not  easy,  but  it  worked. 


UNDERLYING  THEORY 

Student  creation  of  multimedia  fits  well  with  current  trends  in  educational  reform  and  findings  in 
pedagogical  research.  These  approaches  include  learning  by  doing,  group  work  and  cooperative 
work,  multisensory  learning,  learning  by  constructing  knowledge,  performance  based  learning,  and 
portfolio  assessment.  These  types  of  activities  can  be  and  have  been  done  without  things  like 
computers,  but  it  appears  that  students  and  teachers  benefit  from  the  use  of  technology. 

Today's  multimedia  technologies  enable  students  to  perform  activities  that  are  motivated  by  the 
following  theories.  Students  learn  best  if  they  create  their  own  representations  of  knowledge,  and 
interact  with  it,  rather  than  passively  receiving  representations  created  by  others.  Additional  ad- 
vantages exist  if  these  representations  employ  multiple  sensory  modalities,  including  not  only  audio, 
but  also  different  applications  of  vision  such  as  text,  graphics,  images,  animation,  and  video.  Some 
theorists  claim  that  different  students  benefit  from  different  combinations  of  the  various  sensory 
modalities,  so  providing  for  multiple  media  allows  more  students  to  participate  with  excellence. 
Interacting  actively  with  material,  both  during  and  after  creation  of  its  representations,  is  critical. 

Hypertext  links  among  different  parts  of  the  material  can  represent  the  wide  variety  of  relationships 
that  can  exist  among  different  pieces  of  information.  For  examples,  a  Unk  can  run  from  a  cause  to 
one  of  its  effects,  from  a  general  idea  to  a  specific  illustration  of  it,  from  a  general  discussion  to  a 
more  detailed  explanation,  from  one  example  of  a  concept  to  other  examples  of  the  same  concept, 
or  from  an  opinion  to  a  contrary  opinion.  Students  find  these  links  most  useful  if  they  create  the 
links  themselves  to  represent  relations  that  they  have  discovered  and  want  to  record. 

This  is  the  theory  but  what  do  we  do  in  the  classroom?  Actual  achievement  of  the  pedagogical 
benefits  requires  teachers  to  move  beyond  the  function  of  the  teclinologies  and  address  the  issues 
of  articulating  the  goals,  formulating  a  workable  process,  and  assessing  the  results.  ExpUeitly  stating 
the  desired  goals,  paying  close  attention  to  process  and  its  results,  and  producing  continual  im- 
provement are  all  supported  by  the  principles  of  good  instructional  design  as  well  as  by  common 
sense.  We  concluded  that  educators  can  help  students  to  achieve  desired  pedagogical  goals  by  de- 
fining the  goals,  helping  students  to  construct  suitable  multimedia  projects,  and  assessing  the  re- 
sultant projects  against  the  goals. 


THE  PROCESS 
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Tliis  section  briefly  describes  a  process  or  framework  containing  the  steps  thai  teachers  and  students 
can  perform  to  benefit  from  constructing  multimedia.  It  then  gives  examples  of  two  of  the  steps. 
Subsequent  sections  illustrate  these  concepts  with  reference  to  four  student  projects. 

This  process  description  is  intended  only  as  a  guide.  Teachers  will  certainly  vary  what  they  do 
based  on  circumstances  and  experiences  and  their  own  style  of  teaching.  We  sec  the  process  as 
consisting  of  four  steps:  preparation,  assignment,  creation  and  reflection. 

The  preparation  step  requires  the  teacher  to  consider  goals,  topic,  and  organizations  for  the  project. 
She  or  he  must  also  thinking  about  the  grouping  of  students,  what  is  to  he  required  Ccompulsories" 
in  Olympic  parlance)  and  what  would  be  the  characteristics  of  outstanding  projects. 

The  step  of  actually  assigning  the  project  involves  announcing  topic,  educational  goals,  and  exhib- 
its. The  teacher  may  or  may  not  furnish  an  organizing  principle,  template,  or  'sior>'board'.  One 
tactic  is  for  the  class  as  a  whole  to  generate  the  project  structure  and  then  have  individuals  or  groups 
work  on  assignments.  The  assignment  step  alsv^  requires  the  teacher  to  indicate  the  sources  of 
materials  in  various  media  and  specify  the  project  deadlines. 

The  creation  step  is,  naturally  enough,  when  the  students  create.  It  may  include  a  cycle  when  the 
group  members  bring  together  materials,  make  a  plan,  develop  sample,  hold  a  group  critique,  listen 
for  comments,  determine  additional  work,  and  so  on.  The  cycle  may  be  repeated  in  a  larger  group. 
For  example,  the  whole  class  can  discuss  extra  inter-connections  and  build  new  links.  The  final 
exhibition  is  when  the  class  uses  and  enjoys  thcii-  products. 

The  final  step,  which  proceeds  smoothly  from  the  prior  one,  is  for  reflection  and  assessment.  It  is 
when  students  arc  encouraged  to  reflect  on  their  work  and  their  contribution.  The  teacher  reflects 
on  student  performances  hut  also  on  the  project  process  itself  Assessment  can  include  the  class 
evaluation,  public  performances,  interactive  multimedia  built-in  testing,  an<l  Olympic  games-type 
scoring.  If  the  student  and  the  teacher  can  make  the  assumption  that  there  will  be  a  next  time,  then 
criticisms  can  be  cast  in  the  form  of  ideas  for  improvement  in  the  next  project. 

There  is  a  great  deal  that  could  be  said  about  each  of  the  above  steps  and  its  sub-steps.  Ijci  dis- 
cussions of  an  early  sub-step  and  the  last  step  suffice. 

In  the  '^Feacher  assigns  project"  step  of  the  process  a  teacher  must  select  the  groups  of  students 
who  will  work  together.  Practical  experience  has  indicated  that,  unlike  some  other  uses  of  com- 
puters in  education,  students  genuinely  benefit  from  constructing  multimedia  projects  in  groups  that 
range  in  size  from  three  to  five.  The  group  size  that  is  optimal  for  a  given  project  depends  on  the 
natural  number  of  roles  that  students  can  play  within  a  group  wliile  creating  the  project  product. 
Limitations  on  the  availability  of  equipment  usually  bias  the  number  of  students  in  a  group  toward 
the  higji  end  of  the  optimal  range.  Assigning  the  project  to  a  group  gives  teachers  the  opportunity 
to  allow  individual  students  to  develop  specialties,  but  also  gives  the  challenge  of  making  sure  that 
each  student  eventually  plays  each  of  the  several  possible  roles. 

The  ''Reflection  and  assessment*  step  applies  feedback  across  an  entire  step,  in  addition  to  reflection 
that  occurs  at  each  step  as  part  of  deciding  when  to  go  on  to  the  next  step.  ([Constructing  muUi- 
media  projects  gives  students  the  opportunity  to  learn  more  thoroughly  by  actively  organizing  and 
manipulating  information.  Similarly,  applying  objective  criteria  to  such  projects  gives  teachers  an 
opportutiity  to  evaluate  students'  learning  more  authentically.  Some  useful  assessment  criteria  are 
as  follows. 

-  Teachers  should  be  careful  to  measure  what  they  want  students  to  improve,  (especially  in  this 
step,  teachers  must  keep  in  mind  the  academic  goals  that  are  the  real  purposes  of  doing  each 
multimedia  project. 
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-  Serious,  defensible  assessment  is  necessary  in  order  to  move  creating  projects  from  the  role  of  a 
fun  thing  to  do  to  the  role  of  a  standard  part  of  the  curriculum. 

-  Teachers  should  strive  for  repealable,  explainable,  objective  assessments;  but  boih  teachers  and 
students  must  understand  that  authentic  assessment  is  always  more  subjective  than  a  percent  correct 
figure  for  a  multiple  choice  test.  A  school  is  likely  to  need  the  equivalent  of  an  appeals  court  or 
an  ombudsman  to  handle  cases  where  students  feci  that  an  assessment  lias  been  unfair. 

-  Teachers  must  often  assess  a  collection  of  projects  created  by  diflcrent  groups,  in  order  to  assess 
the  performance  of  each  individual.  In  particular,  individual  assessment  may  require  overt  inter- 
ference in  the  groups,  to  make  sure  that  each  individual  eventually  plays  all  roles.  Some  special- 
ization is  desirable,  but  both  learning  and  assessment  require  that  specialization  must  not  be 
overdone. 

-  Teachers  must  be  aware  of  the  dark  side  of  decorative  media:  these  can  distract  both  students  and 
teachers  from  serious  content.  Teachers  must  make  sure  that  the  'compulsorics"  arc  completed  and 
the  media  must  illustrate  mastery  of  the  content,  not  mask  lack  of  understanding. 

Multimedia  is  fundamentally  a  mode  of  communication.  Teachers  must  check  that  each  project 
actually  communicates  the  important  information  to  the  relevant  audience,  be  it  the  teacher,  peer 
students,  or  a  wider  audience.  Assessment  can  help  students  to  improve  their  use  of  different  media 
to  convey  different  types  of  infonnation  most  effectively. 


THE  FOUR  PRO.IECTS: 

''Current  Events"  project  with  hypertext,  optional  scanned  photographs 

The  'Current  Events'  project  is  a  multimedia  version  of  the  standard  assignment  to  bring  in 
newspaper  articles,  post  them  on  a  bulletin  board,  and  connect  related  articles  with  colorful  tapes. 
Different  colors  distinguish  different  types  of  relationships  among  articles. 

The  multimedia  project  replaces  the  bulletin  board  by  a  computer  disk.  Each  student  summarizes 
an  article  on  one  page  of  a  multimedia  folder,  optionally  adds  a  scanned  picture,  and  tlien  creates 
hypertext  links  from  the  article  to  other  articles  and  to  such  things  as  timelines,  maps,  concept 
maps,  and  word  definitions.  The  result  of  this  project  is  a  continuously  expanding  multimedia 
database  that  is  available  for  students  to  navigate  through,  to  find  specific  information. 

The  technology  and  the  multimedia  goal  framework  of  this  project  enhance  the  articles'  pedagogical 
value  by  focusing  on  in  depth  understanding  of  the  articles  and  of  their  connections  to  other  articles 
and  to  concepts.  Related  contributers  to  increased  learning  include  the  sustained  attention  to  a 
topic  over  time,  paying  attention  to  differing  points  of  view,  independent  but  cooperative  work, 
articulating  connections  between  different  points  of  view,  determining  multiple  contexts,  and  at- 
tention to  potential  sources  of  bias. 

A  teacher  selects  a  domain  or  subject  for  the  newspaper  articles,  depending  on  the  teacher's  edu- 
cational objectives  and  the  students'  interests.  The  teacher  should  consider  suggesting  articles  from 
ethnic  newspapers,  including  non-English  newspapers  in  the  case  of  language  classes,  where  sum- 
maries need  not  be  in  the  same  languages  of  the  articles.  The  teacher  should  insist,  as  one  of  the 
compulsories  (using  the  Olympic  games  terminology)  that  each  article  include  a  citation,  a  link  to 
at  least  one  other  article,  and  a  link  to  both  a  timeline  and  a  map.  An  excellent  contribution  to  this 
project  could  include  also  creating  a  concept  map  and  adding  links  from  nodes  of  that  map  to  many 
relevant  articles,  adding  a  scanned  picture  or  graphic  to  the  summary,  and  linking  difilcult  words 
to  their  definitions  in  a  growing  class  dictionary. 
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The  teacher  has  to  be  prepared  to  answer  questions  about  why  the  project  einphasi/es  text  sum- 
maries of  articles,  augmented  by  only  an  occasional  scanned  picture  or  graphic,  rather  than  scanning 
in  the  actual  articles.  The  answer  is  that  a  fairly  dense  screen  containing  a  lyped  summary  can  be 
stored  in  under  one  thousand  bytes  of  disk  space,  whereas  a  scanned  image  of  a  newspaper  artick:, 
with  sufficiently  high  resolution  to  be  readable,  can  occupy  one  million  by!i\s  of  space.  With  text 
summaries,  a  given  computer  can  store  a  thousand  times  more  typed  summaries  than  scannccl  ar- 
ticles. Tor  example,  a  moderately  large  hardfile  would  be  filled  by  about  100  scanned  images  of 
articles,  but  could  hold  one  hundred  thousand  text  summaries.  Technically,  typing  in  a  text  sum- 
mary is  a  wonderfully  effective  way  of  compressing  an  image.  Of  course,  Optica!  (Character  Re- 
cognition is  a  possibility.  But  then,  student  generated  summaries  also  can  provide  a  higher  order 
thinking  skill  goal  as  well. 

In  the  'Reflection"  step  of  the  process,  teachers  should  look  for  ways  to  encourage  improvement 
in  the  richness  of  the  connections  among  articles,  incrementally  adding  to  the  assignment  such 
things  as  supporting  analysis  essays  and  factual  quizzes  on  the  contents  of  the  archive.  With  this 
continual  follov/-up  the  archive  will  grow  ana  becomes  more  useful  and  interesting. 


Xrities'  Circle"  project  with  digital  audio,  photographs,  graphics 

This  project  can  be  applied  many  times  to  many  different  subjects.  Their  common  element  is  the 
opportunity  to  record  and  analyze  difference  of  opinion  about  the  subject.  We  applied  ''Critics' 
Circle*  to  opinions  about  a  movie.  The  same  techniques  can  be  applied  later  to  serious  political, 
social,  ethical,  or  even  technical  issues. 

'Hie  teacher  divides  the  class  into  groups  of  five  students.  Each  group  prepares  a  multimedia  folder 
that  records  three  orally  expressed  points  of  view  on  the  movie.  For  each  person  who  records  an 
opinion,  the  project  includes  either  a  scanned  or  frame  grabbed  picture  or  a  graphic  sketch,  to  be 
displayed  while  that  person's  opinion  is  being  played  back.  The  group  also  provides  an  introduc- 
tory page  from  which  a  reader  can  select  any  of  the  recorded  opinions  and  prepares  a  (formal) 
written  summary  of  the  differences  of  opinion  that  were  expressed  using  the  (in  forma!)  oral  me- 
dium. 

Creating  this  project  includes  selecting  a  subset  of  the  group's  views  that  represent  the  spectrum 
of  opinion  and  that  are  individually  and  collectively  interesting  and  entertaining,  and  that  can  be 
compelling  expressed  using  a  combination  of  text,  graphics  and  audio.  The  project  requires  that 
the  chosen  ^critics"  be  named  or  characterized  and  the  clash  or  relationship  of  opinions  be  de* 
scribed.  The  group  members  must  come  to  terms  with  the  experience  of  different  roles.  In  some 
situations,  members  will  consider  role-playing  to  present  a  view  different  from  their  own. 

Rach  of  the  projects  can  be  used  to  achieve  a  particular  list  of  goals  of  the  three  types  discussed 
above.  For  "Critics'  Circle*  the  goals  are  as  follows: 

Higher  order  thinking  skills 
Working  successfully  in  a  group 
Making  choices  in  a  group 

Recognizing  spectrum  of  views  and  presenting  them  in  a  meariingful  way 
Irna^nation  and  empathy 
Ability  to  summarize 
Role  playing 
Content 

What  different  people  react  to  in  a  given  topic  or  movie 
Deeper  understanding  of  intrinsic  content  of  topic 
Dramatic  /  production  values  in  films 
Icchnical 
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Basic  authoring  of  folders;  linking  lo  provide 
coherent  and  logical  organization  of  materials  on 
opinions  of  the  movie 
Creation  of  graphic  buttons  (icons) 
Use  of  and  creation  of  pictures 
Use  of  and  creation  of  audio  recording 
Total  effect  of  text,  graphics,  and  audio 


Trailers'  project  with  control  of  commercial  video,  recorded  audio 


Each  of  several  groups  creates  a  trailer  for  a  given  movie.  A  trailer  is  a  short  sSct  of  clips  and 
voice-overs  that  the  group  designs  in  order  to  make  a  given  audience  want  to  go  sec  that  movie. 

This  project  is  organized  as  a  specific  tour  that  the  authors  prepare  in  order  to  have  a  particular 
effect  on  the  selected  audience,  so  user-driven  navigation  through  hypermedia  gets  to  take  the  day 
off. 


Technically,  the  tour  consists  of  a  sequence  of  instructions  that  cause  a  laserdisc  player  to  play 
particular  video  clips,  synchronized  with  instructions  to  replay  digitally  recorded  voiccovers.  Iwcn 
simple  multimedia  authoring  tools  provide  good  ways  for  students  to  prepare  such  a  tour.  How- 
ever, tools  differ  significantly  in  the  amount  of  effort  they  require.  The  less  capable  tools  require 
students  to  time  the  lengths  of  their  digital  audio  recordings,  and  their  selected  video  clips,  and  then 
to  tell  the  system  how  long  to  wait  for  playing  to  finish,  in  what  looks  suspiciously  like  a  pro- 
gramming language.  More  capable  tools  represent  each  segment  of  digital  audio  and  each  video 
clip  by  a  visible  icon,  and  allow  a  student  lo  prepare  a  tour  by  simply  dragging  icons  into 
juxtaposition  along  a  timeline. 

Teachers  must  be  prepared  to  answer  a  question  such  as  'Why  must  this  project  use  a  computer- 
controlled  laserdisc  player  when  Video  Cassette  Recorders  and  their  media  are  so  much  more  widely 
available?*  The  answer  is  that  a  direct-access  medium,  such  as  a  laserdisc,  can  find  the  next  desired 
clip  in  less  than  a  second,  by  moving  a  head  radially  across  the  tracks  of  the  disc.  Contrast  that  to 
a  VCR  which  (even  if  controlled  by  computer)  may  require  several  minutes  to  find  a  desired  clip 
by  passing  over  all  of  the  intervening  tape.  People  can  maintain  a  train  of  thought  during  a  wait 
that  lasts  a  second  but  not  during  a  wait  that  lasts  several  minutes 


'Planning'  project  with  organization  of  materials,  getting  started 


Rach  group  prepares  a  multimedia  presentation  on  its  plans  for  using  multimedia  in  a  particular 
discipline,  with  emphasis  on  how  the  information  is  organized.  For  students  who  were  not  actually 
teachers,  this  project  would  have  been  directed  to  presenting  some  content  from  a  discipline  selected 
by  each  group. 

In  this  project,  the  teacher  discusses  several  examples  of  projects  that  use  different  organizations 
of  material.  TTic  examples  range  from  work  by  4lh  graders  and  7th  graders  to  fairiy  professional 
materials.  Each  group  was  asked  to  prepare  a  muhimedia  presentation  on  their  plans  for  the 
coming  year,  with  buttons  on  schedules  and  milestones,  and  with  possible  roadblocks  identified. 
They  then  give  the  presentation  aiid  discuss  their  plans. 


THE  WORKSHOP 


We  had  decided  to  teach  student  creation  of  multimedia  projects  to  teachers  by  example,  that  is, 
by  having  them  pretend  lo  be  their  students  and  actually  create  a  sequence  of  multimedia  projects 
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of  increasing  difficulty.  Wc  did  not  know  whether  teachers  could  do  this,  in  a  strange  environment 
of  hardware,  software,  and  multiple  media,  and  also  be  able  to  think  about  how  they  would  sub- 
sequently coach  their  own  students  in  doing  similar  projects.  To  make  success  more  likely,  wc  split 
the  workshop  into  three  parts  and  provided  templates  for  tlie  teachers  to  use. 

The  workshop's  parts  were  as  follows. 

-  In  early  June,  1992,  we  spent  a  day  introducing  the  authoring  system  and  the  four  projects  to  a 
half  dozen  technical  support  staff.  For  this  session,  wc  set  up  two  multimedia  computer  systems 
in  a  conference  room  in  IBM  Watson  Research  Center's  Hawthorne  I^b. 

-  One  week  later,  we  held  an  orientation  day  for  all  of  the  teachers  and  staff  people  in  the 
Yorktown  Heights  Lab.  In  the  morning,  we  presented  the  framework  and  projects  described  above 
to  one  half  of  the  group,  in  a  large  room  equipped  with  a  multimedia  system  and  a  large  overhead 
projector.  Next  door,  the  other  half  of  the  group  used  a  dozen  machines  that  had  no  special 
hardware,  to  work  througli  the  online  tutorial  that  is  a  part  of  our  selected  multimedia  authoring 
system.  The  halves  reversed  roles  in  the  afternoon.  Staff  people  helped  make  sure  that  everybody 
finished  the  tutorial  successfully  and  also  took  video  pictures  of  the  teachers  for  them  to  use  later 
in  the  Xrities'  Circle"  project. 

-  I'inally,  in  mid-July,  the  teachers  and  staff  people  spent  a  week  at  the  Hawthorne  I.ab,  where 
we  had  set  up  seven  fully-equipped  multimedia  systems  in  different  small  rooms  for  their  use.  We 
started  each  day  with  a  brief  lecture  and  longer  group  discussion.  Then  the  teachers  broke  into 
groups  and  created  one  of  the  projects.  Finally,  the  groups  came  back  together  and  presented  their 
results.  We  allocated  an  entire  day  to  creating  the  third,  Trailers,''  project,  and  held  its  presenta- 
tions the  following  morning. 


We  provided  the  workshop  attendees  with  templates  for  three  of  the  projects.  A  template  is  an 
example  of  a  usable  project,  on  a  diskette,  that  the  teachers  could  modify.  T^specially  for  the  first 
few  projects,  using  a  template  is  an  attractive  alternative  to  starting  from  scratch  to  create  a  project. 
In  a  sense,  templates  constitute  their  own  layer  in  the  software  hierarchy,  just  above  j^ulhoring  tools 
and  reference  materials,  but  belov/  urunodifiable  professionally  created  titles. 

The  workshop's  first  three  projects  form  a  sequence,  starting  with  a  quite  complete  template,  in 
which  students  create  their  first  project  by  simply  doing  more  of  what  the  template's  author  did, 
and  ending  with  a  completely  optional  template,  where  students  are  encouraged  to  look  at  an  ex- 
ample and  then  do  their  own  thing  for  the  third  project.  For  the  fourth  project,  wc  just  showed 
several  examples,  but  handed  out  I'lo  diskettes.  The  primary  purpose  of  providing  a  complete 
template  for  early  projects  is  to  allow  students  to  concentrate  most  of  their  attention  on  the  subject 
matter,  rather  than  struggling  with  the  details  of  operating  the  software.  Secondarily,  of  course, 
templates  also  teach  by  example.  A  complete  template  includes  a  printed  *cook-book'^  telling  what 
keystrokes  and  mouse  motions  are  involved.  These  are  the  instructions,  that  everyone  reads  only 
when  all  else  fails.  Of  course,  actual  templates  and  cook-books  are  specific  to  one  authoring  tool, 
but  the  concept  has  general  applicability. 


RliSULTS  OP  THE  WORKSHOP 

This  workshop  was  considered  a  success  by  the  participants,  the  staff  of  the  NYC  Public  School 
Technical  Assistance  Centers,  and  the  instructors,  who  learned  as  much  as  the  attendees  about  the 
reality  of  the  classroom.  We  will  touch  on  only  a  fraction  of  the  issues  which  surfaced,  including 
the  wide  variety  of  teacher  experience,  the  challenges  of  scheduling  time  and  space,  the  usefulness 
of  completed  projects,  and  the  role  of  competition  among  students  and  among  students  and 
teachers. 

We  had  expected  that  the  circumstances  of  this  workshop  would  mean  the  attendees  would  be 
self-selected  and  all  would  have  considerable  experience  with  computers.  We  found  instead  a  great 
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variety:  even  today  (here  are  many  teaehers  with  little  eomputer  experienee  and  perhaps  even  some 
eomputer  pliobia.  We  think  that  by  the  end  ot  the  week,  througii  the  experienees  and  the  gentle 
introduetions,  everyone  was  eomfortable  and  eager  to  do  more.  Honesty  prevailed.  At  the  end, 
one  of  the  teaehers  said  ""l  think  now  I'm  ready  to  do  the  tutorial  again".  It  is  important  to  make 
sure  that  even  the  beginners  benefit. 

At  the  start,  we  realized  that  potentially  there  were  three  eategories  of  "problem'  workshop  parlie- 
ipants:  teachers  v/ho  were  not  eomputer  literate,  perhaps  even  still  eomputer  phobie;  the  opposite, 
that  is,  teaehers  who  were  already  familiar  with  multimedia  and  who  might  thus  resent  a  slow  paee; 
and  people  who  had  strong  allegiances  to  other  products  and  product  lines.  We  were  quite  gratified 
that  at  the  close  of  the  workshop,  all  of  the  participants,  including  the  technical  staff,  expressed 
strongly  positive  assessments  about  the  workshop,  the  method  of  teaching,  and  even  of  the  products 
used. 

We  were  especially  gratified  that  the  minority  teachers  indicated  that  they  felt  that  multimedia 
projects  would  especially  benefit  the  inner-city  children.  Tlie  teachers  and  staff,  were  committed 
to  overcoming  obstacles  of  budget,  time  and  bureaucracy  to  implement  these  programs  in  their 
schools. 

Scheduling  of  time  and  place  remain  a  problem,  either  in  a  school  or  in  a  corporate  research  lab. 
Time  must  be  scheduled  in  long  enough  blocks  to  allow  sustained  creative  activity.  Hquipment 
must  be  placed  close  together  so  that  one  person  can  coach  and  advise  several  groups,  yet  use  of 
sound  in  most  projects  favors  some  separation  of  the  groups  from  one  another  and  from  class  dis- 
cussions. Student  created  multimedia  projects  can  become  important  parts  of  the  teaching  and 
learning  process  only  if  the  projects  are  integrated  into  the  general  curriculum  of  various  disciplines 
and  combinations  of  disciplines,  rather  than  being  separated  out  into  computer  literacy  classes. 
This  works  far  better  if  the  equipment  is  available  in  the  regular  classrooms,  rather  than  being  re- 
legated to  a  computer  lab  or  library  audio-visual  center. 

One  teacher  brought  up  the  disadvantage  that  a  completed  multimedia  project  can  be  enjoyed  only 
by  people  who  have  the  right  technology  available,  unlike  reports  written  using  paper  and  pencil, 
that  anyone  can  read.  Several  of  the  teachers  countered  that  the  pencil  and  paper  reports  were  so 
uninteresting  that  no  one  read  them,  whereas  people  would  go  out  of  their  way  to  sec  a  multimedia 
report.  They  emphasized  that  the  expectation  of  an  audience  strongly  encouraged  students  to  im- 
prove their  results. 

A  surprise  at  the  workshop,  though  the  instructors  should  have  expected  it,  was  the  negative  re- 
action to  voting  on  the  best  Trailers''  project.  The  discussion  divided  into  two  distinct  issues: 
competition  among  young  students  and  competition  among  teachers.  The  instructors  believe  that 
competition  has  its  place  in  school,  because  the  fact  is  that  there  is  much  competition  in  life.  The 
advantages  of  distinct  competitions  among  well-chosen  groups  is  that  the  students  know  the  rules 
and  they  can  come  back  to  compete  another  day.  In  contrast,  for  many  children  much  of  the 
competition  is  subtle  and  on-going  and  they  can  never  win.  However,  it  is  our  opinion  that  the  real 
issue  at  the  workshop  was  the  second,  competition  among  teachers  themselves.  These  teachers^  like 
many  others,  feel  they  arc*  constantly  being  judged  and  it  Is  often  unfair.  Though  we  saw  this  as  a 
'fun'  thing,  and  did  not  think  there  was  an  obvious  winner^  that  is,  practically  all  the  projects  were 
contenders,  a  significant  portion  of  the  attendees  preferred  to  pass  on  the  voting. 

We  feel  that  we  validated  the  concept  of  using  student  created  multimedia  projects  to  meet  aca- 
demic goals  and  also  validated  the  approach  of  teaching  the  concept  to  teachers  by  means  of  ex- 
amples. We  arc  continuing  to  receive  feedback  from  the  teachers  and  would  appreciate  comments 
from  readers. 
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ABSTRACT 

This  paper  seeks  to  describe  the  benefits  associated  with  a  programmatic 
approach  to  educational  technology  research.  Session  participants  will  be  provided 
with  an  example  of  a  programmatic  approach  which  is  being  employed  to  address 
limitations  in  existing  media  comparison  and  replication  research.  The  paper 
describes  some  fundamental  practical  problems  in  conducting  research  on 
technology  and  will  employ  specific  examples  from  studies  in  the  application  of 
interactive  technology  to  social  learning  to  illustrate  key  points  and  suggest  ways  in 
which  problems  can  be  addressed. 
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Concerns  about  the  source  of  achievement  effects  have  emerged  as  a 
frequently  discussed  issue  in  technology  research  (Clark,  1983;  Ko/ma,  1991; 
Hassclbring  el  al,  1987-88;  Reeves,  1986).  Much  of  the  conjecture  is  based  upon 
the  observation  that  many  media  comparison  studies  are  confounded  by  design 
weaknesses,  which  in  part,  result  from  a  failure  to  consider  and/or  control  for  rival 
hypotheses  (Kearsley  &  Frost,  1985).  A  conclusion  which  is  commonly  drawn  is 
that  the  critical  variable  in  media  comparison  studies  is  not  the  medium  but 
associated  factors  such  as:  instructional  design  (Hasselbring  et  al,  1987-88). 
treatment  implementation  effects,  the  uncontrolled  effects  of  novelty  associated 
with  the  application  of  new  technology  (Clark,  1983),  and  effects  associated  with 
the  teachers  engaged  in  implementing  the  interventions.  These  explanations, 
amongst  others,  have  been  used  to  account  for  the  results  of  studies  which  have 
reported  support  for  their  respective  hypotheses,  but  have  failed  to  control  for 
sources  of  bias,  Clark  (1983)  strongly  suggested  that  given  these  problems,  studies 
involving  media  comparisons  are  inadvisable  and  that  medium  is  not  a  significant 
factor  in  explaining  achievement  outcomes.  More  recent  work  by  Kozma  (1991) 
challenges  this  assertion  by  suggesting  that  learning  requires  an  interaction  between 
the  characteristics  of  the  learner  and  the  medium  which  can  exert  an  inlluence  on 
cognitive  processes  and  subsequent  learning,  Kozma  suggests  that  rather  than 
dispense  with  media  comparison  research,  the  nature  of  the  media-learner 
interaction  should  be  subjected  to  further  investigation,  A  programmatic  approach 
may  be  required  in  order  to  undertake  a  full  exposition  of  this  relationship,  vSuch  an 
approach  would  seek  to  systematically  link  theory,  content,  teaching  and  learning. 

Over  the  past  six  years  we  have  been  conducting  a  very  practical 
investigation  of  the  type  of  social  skills  which  are  problematic  for  adolescent-aged 
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students  and  their  teachers  in  school  settings  and  the  role  of  interactive  video  in 
assessing  and  teaching  those  skills  (e.g.,  Bain,  1990;  Bain,  Houghton  &  Farris, 
1991,  Carroll,  Bain  &  Houghton,  1992;  Bain,  Houghton,  Foo  &  Carroll,  1992).  It 
should  be  noted  that  throughout  the  paper  the  pronoun  "we"  will  be  used  to 
acknowledge  the  input  of  my  co-researchers  to  the  research  program. 

To  date  the  project  has  involved  eight  related  studies,  one  of  which  is 
currently  in  pnDgrcss  which  have  examined  factors  associated  with  the  application 
of  interactive  video  technology  to  social  learning.  Our  research  has  sought  to  link 
theory,  content,  and  the  characteristics  of  teacher/learner  and  teaching/learning 
interactions  in  order  to  provide  meaningful  conclusions  about  the  role  of  interactive 
media  in  social  learning.  This  work  has  been  undertaken  at  the  University  of 
Western  AusU'alia  and  Lehigh  University  with  involvement  from  faculty  at  Western 
Michigan  University.  The  project  has  resulted  in  the  development  of  a  28  lesson 
curriculum  which  employs  the  HyperCard  and  HyperCard  II  programming 
languages.  Animated  sequences  and  graphics  included  in  the  lessons  were 
developed  using  Macromind  Director  and  Supercard  programs.  The  program  is 
approximately  30  megabytes  in  size  and  includes  2  hours  of  disc-based  video 
material.  The  program  is  a  level  three  interactive  video  application  which  runs  on  a 
Macintosh  I!  computer  incorporating  a  MASS  colorspace  video  board  and  linked 
with  a  Sony  3600D  multistandard  video  disc  player. 

While  for  the  purposes  of  this  presentation,  it  is  not  possible  to  describe  in 
full  detail  the  methodology  and  results  associated  with  each  of  the  studies 
undertaken,  the  ongoing  nature  of  the  project  has  provided  us  with  what  we  believe 
to  be  some  important  insights  into  the  difficulties  associated  with  drawing 
meaningful  conclusions  about  the  role  of  media  in  instruction  and  has  provided  a 
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basis  for  the  continued  investigation  of  the  role  of  interactive  technology  in  teaching 
and  assessment  in  the  field  of  social  skills.  The  purpose  of  this  paper  is  to  share 
those  insights  with  you. 

The  paper  will  focus  directly  on  a  number  of  problems  we  believe  to  be 
characteristic  of  research  involving  the  application  of  technology  to  instruction.  I 
will  describe  the  ways  in  which  those  problems  were  addressed  in  undertaking  our 
own  research.  Our  studies  and  their  findings  will  be  used  to  illustrate  our  approach 
and  to  serve  as  evidence  for  general  recommendations  regarding  the  conduct  of 
future  research  programs. 
Problem  1 

The  a^^piiwation  of  technology  to  instruction  is  commonly  based  on  its 
perceived  potential  as  a  solution  to  an  applied  problem.  Unfortunately,  research 
often  proceeds  without  recognition  of  important  theoretical  and  applied  research 
which  may  have  a  critical  bearing  on  the  long  term  outcomes  of  a  project.  There  is, 
in  many  instances,  a  failure  to  provide  an  adequate  theoretical  link  between  the 
technological  medium,  teaching  and  learning,  and  the  application.  This  is  a 
fundamental  construct  validity  question  which  is  commonly  left  unattended. 
Research  commonly  proceeds  on  the  basis  of  intuition  and  assumption. 

Our  own  research  is  underpinned  by  the  extensive  body  of  research  on 
social  learning,  particularly  the  work  of  Albert  Bandura  and  his  multiprocess  theory 
of  observational  learning.  However,  the  project,  like  many  others,  began  as  as  a 
response  to  a  practical  problem.  In  teaching  social  skills  to  young  people  we  found 
that  their  capacity  to  obtain  benefit  from  the  instruction  seemed  limited  by  our 
capacity  to  adequately  depict  social  situations  in  a  manner  which  was  perceived  by 
adolescent-aged  students  to  be  be  meaningful  and  impactful.   Students  with 
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attentional  difficulties  for  example  have  significant  needs  in  the  areas  of  social  skills 
(Sagvolden  &  Archer,  1989)  and  commonly  have  difficulty  in  connecting  their 
behavior  with  the  social  contexts  in  which  they  function.  There  was  a  clear  intuitive 
and  inviting  precedent  and  rationale  for  the  application  of  video  technology  to  this 
area  in  order  to  depict  interesting  social  situations  which  would  be  hard  to  describe 
in  class  discussion  or  recreate  in  roleplay. 

However,  before  pursuing  information  on  the  role  of  technology,  we 
sought  to  identify  links  between  the  nature  of  social  learning  and  the  potential  role 
of  media  in  that  learning.  Bandura's  social  learning  theory  (1986)  provided  the 
theoretical  rationale  and  a  guide  to  the  application-  By  referencing  our  intuition  to  a 
theoretical  perspective  we  opened  our  project  up  to  a  wealtli  of  research  on  the  way 
people  learn  from  models,  the  types  of  models  which  are  most  effective  and  the 
cognitive  processes  which  translate  observation  into  behavior.  For  example,  we 
found  studies  on  the  effects  of  gender,  model  type  (positive  or  negative),  peer 
versus  adult  models  and  model  attractiveness.  From  Bandura's  theory  we  leanied 
many  things  about  the  way  video  could  be  most  effectively  deployed  to  assist  with 
increasing  attention  to  models  and  to  serve  as  a  basis  for  the  rehearsal  and 
production  of  social  behavior.  We  also  used  previous  studies  in  the  areas  of  social 
learning  and  modeling  to  articulate  the  type  of  content  to  be  developed  and  the  way 
in  which  it  would  be  presented,  including  some  very  practical  considerations 
associated  with  the  age  of  the  kids  who  would  participate  and  even  what  kids 
should  wear  for  our  video  sequences. 

In  order  to  further  develop  the  theoretical  rationale  for  our  own  research,  we 
investigated  the  effects  of  model  type  and  linear  and  interactive  video  medium  on 
the  attention  and  recall  of  early  adolescents  with  attentional  difficulties.  This  study 
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provided  us  with  additional  information  on  the  effects  of  model  type.  We  found  for 
example,  that  juxtaposing  positive  and  negative  video  models  resulted  in  higher 
levels  of  comprehension  than  a  positive-neutral  model  sequence.  We  also  found  that 
attention  and  comprehension  were  statistically  significantly  higher  for  interactive 
video  when  compared  to  linear  video,  findings  which  provided  us  with  the 
empirical  basis  to  employ  interactive  over  linear  video  in  the  development  of 
ti'aining  materials. 

In  summary,  our  first  recommendation  is  simply,  attend  to  construct 
validity.  Generate  a  theoretical  link  between  the  suppositions  and  assumptions 
associated  with  the  application  of  technology  to  instruction  and  use  preceding 
research  to  identify  key  considerations  in  implementation  as  well  as  to  provide  a 
SO':. id  theoretical  basis  for  the  work. 
Problem  2 

The  content  of  the  application  needs  to  be  validated  with  respect  to  the  population 
with  whom  it  will  be  used.  To  what  extent  is  the  content  of  the  application  relevant 
to  tlie  population  of  interest? 

Our  second  problem  pertains  to  the  interaction  between  the  theory,  the 
population  of  interest  and  the  content  of  the  application.  Where  one  is  confident 
about  the  relationship  between  theory,  medium  and  the  application,  a  need  exists  to 
subject  the  content  of  the  application  to  a  validation  which  seeks  to  link  theory, 
application  and  target  population.  It  is  at  this  point  that  we  recognize  the  interaction 
between  the  needs  of  a  population  and  the  application.  Again,  with  respect  to  our 
own  project,  the  wealth  of  research  in  the  area  of  social  learning  provided  us  with  a 
foundation  on  which  to  build.  We  also  recognized  the  more  fundamental  need  to 
validate  the  contents  of  any  materials  we  would  develop.  While  a  theoretical 
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rationale  may  exist  for  the  application  in  question,  a  rigorous  examination  of  the 
content  is  necessaiy  if  the  application  is  to  work. 

One  of  our  first  major  studies  sought  to  address  this  issue  by  empirically 
validating  those  social  situations  which  were  problematic  for  early  adolescents  in 
school  settings.  In  an  article  published  in  the  British  journal,  Educational 
Psychology,  we  described  the  application  of  the  Behavior  Analytic  Method 
(Goldfried  &  D'Zurilla,  1969)  to  identify  and  validate  the  content  of  those  social 
situations  which  occur  in  school  and  are  problematic  for  early  adolescents.  In  order 
to  achieve  this  objective  we  reviewed  assessment  devices,  curricula  and  the  social 
skills  literaturf,  to  identify  a  pool  of  relevant  social  skills  and  then  sampled  the 
opinions  of  over  100  students  and  teachers  in  order  to  elucidate,  in  a  disciplined 
manner,  those  skills  and  situations  which  would  serve  as  the  basis  of  our 
assessment  and  cuniculum  materials.  This  research  resulted  in  the  development  of 
a  matrix  of  16,  content  validated  skills  and  situations  which  served  as  the  basis  for 
our  application.  These  situations  and  skills  were  turned  into  video  sequences  which 
in  turn  served  as  the  basis  for  our  assessment  device  and  curriculum. 

Just  as  one  cannot  assume  a  relationship  between  theoretical  constructs  and 
practice,  nor  can  one  assume  the  validity  of  the  content  derived  from  the  theory. 
Our  second  recommendation  is  to  attend  to  content  validity.  Use  a  disciplined 
approach  to  validate  the  content  of  tlie  application. 
Problem  3 

Any  application  of  technology  to  teaching  and  instruction  must  derive  its 
design  from  empirically  supported  principles  of  instruction.  Applications  which 
involve  the  combination  of  instructional  technology  commonly  proceed  on  the  basis 
of  assumptions  about  the  design  of  instruction  which  often  have  little  in  the  way  of 
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empirical  support  regarding  iheir  effects  on  educational  productivity.  It  should  come 
as  no  surprise  that  research  on  the  effectiveness  of  instructional  technology  has 
been  equivocal,  when  one  considers  the  paucity  of  studies  which  have  adequately 
considered  the  instructional  design  variable  and  adequately  controlled  for  factors 
associated  with  the  design  and  implementation  of  instruction. 

For  the  purposes  of  our  most  recent  study  involving  a  comparison  of 
interactive  and  linear  video  we  developed  carefully  scripted  lessons  based  upon  the 
principles  of  Structumd  Learning  and  Direct  Instruction.  The  lesson  formats  were 
piloted  with  early  adolescent  aged  students  as  part  of  a  formative  evaluation 
procedure  and  prior  to  the  development  of  any  video  materials.  We  used  the 
evaluation  to  ascertain  the  extent  to  which  the  design  of  lessons  was  sound  and  the 
delivery  clearly  understood  by  students  who  would  be  the  consumers  of  our 
interactive  video-based  social  problem-solving  curriculum  materials.  When 
studying  the  effects  of  the  curriculum,  we  also  sought  to  establish  the  integrity  with 
which  teachers*  implemented  lessons,  not  only  in  the  interactive  condition  but  in  all 
3  of  the  conditions  under  investigation.  This  involved  reviewing  videotapes  and 
matching  the  observed  behavior  with  the  scripted  sequences  for  each  lesson  (see 
Bain,  Houghton,  Foo  &  Carroll,  1992). 

Our  thitu  recommendation  is  to  pay  strict  attention  to  those  characteristics  of 
effective  instruction  and  design  applications  on  the  basis  of  procedures  and 
practices  which  have  been  shown  to  work. 
Problem  4 

Any  investigation  which  im.plies  an  interaction  of  technology,  teaching  and 
learning  must  address  both  the  processes  of  teaching  and  learning  as  well  as  their 
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products.  Few  studies  have  sought  to  address  these  variables  and  to  replicate 
findings  in  order  to  improve  reliability. 

The  multiplicity  of  factors  which  have  been  shown  to  correlate  with  student 
achievement  and  the  failure  to  explain  achievement  effects  in  IV  research  (McNeil 
&  Nelson,  1991)  would  suggest  a  need  to  consider  the  effects  of  process  variables 
in  media  comparison  studies.  Contemporary  research  on  teaching  has  focused  on 
the  search  for  influential  variables  associated  with  school  learning  which  optimize 
educational  outcomes  across  school  settings  (Wang  &  Walberg,  1991).  For 
example,  the  extensive  body  of  educational  productivity  research  has  recognized  the 
contribution  of  such  process  variables  as:  teacher  feedback  and  encouragement, 
classroom  organizational  factors  and  student  attention,  all  of  which  are  significantly 
correlated  with  learning  outcomes  (Frazer,  Walberg,  Welch  &  Hattie,  1987). 

The  contribution  of  these  variables  to  the  outcomes  of  instruction  and 
educational  productivity  broadens  the  term  of  reference  for  researchers  interested  in 
examining  media  effects,  particularly  if  one  accepts  that  a  substantive  change  in  the 
delivery  of  instruction  also  represents  a  major  change  in  the  stimulus  conditions 
under  which  instruction  is  to  occur  and  as  such,  may  also  influence  achievement- 
related  variables.  For  example,  group  based  interactive  video  instruction  can  alter 
the  role  and  behavior  of  the  teacher  and  students  in  classroom  instruction.  Where 
under  a  more  u-aditional  didactic  model  the  teacher  may  be  totally  responsible  for  the 
delivery  of  content,  when  using  interactive  video  the  teacher  may  share  that 
responsibility  with  the  interactive  video  delivery  system  in  the  form  of  a  video 
narrator  (e.g..  Systems  Impact,  1985),  Such  a  change  may  influence  the  way  in 
which  students  attend  to  the  stimulus  information  presented  and  the  type  and 
amount  of  interaction  which  occurs  between  student  and  teacher  or  student  and 
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interactive  video.  By  including  both  process  and  product  variables  in  research 
studies  involving  technology,  it  may  be  possible  to  obtain  a  clearer  picture  of  the 
effects  of  the  medium  on  student  outcomes  (e.g.,  attitude  and  achievement)  and 
factors  known  to  be  related  to  those  outcomes  (e.g.,  attention,  opportunities  lo 
respond). 

In  our  last  iv/o  studies  we  sought  to  measure  both  process  and  product 
factors  as  part  of  the  investigation.  In  a  study  by  Carroll,  Bain  &  Houghton  (1991) 
which  compared  the  effects  of  interactive  and  linear  video  with  72  children  with 
Attentional  Difficulties,  we  selected  attention  as  a  process  variable  in  addition  to 
measuring  comprehension  of  social  interactions.  We  used  a  direct  observation 
approach  to  measure  the  levels  of  attending  under  both  interactive  video  and  linear 
video  conditions  and  found  a  statistically  significant  effect  in  favor  of  the  interactive 
video  condition.  In  a  second,  more  applied  study  by  Bain,  Houghton,  Foo  & 
Carroll  (1992)  we  conducted  a  comparison  between  interactive,  linear  and  non- 
video  conditions  for  teaching  social  problem-solving  to  early  adolescents  in 
classroom  settings.  The  study  measured  attitude  and  achievement  as  the  product 
variables  and  also  included  measurement  of  teacher  encouragement  and 
questioning,  attending,  lesson  duration  and  student  participation,  all  of  which  have 
been  described  as  process  variables  in  the  educational  productivity  research.  The 
results  indicated  statistically  significant  differences  between  the  conditions  for 
attitude  and  achievement  in  favor  of  interactive  video  with  differential  effects  across 
the  three  conditions  form  the  process  variables.  For  example,  attention  levels  were 
highest  in  the  interactive  video  condition,  although  active  student  participation  was 
lowest.  These  were  interesting  findings  given  the  previously  described  effects  of 
student  participation  on  achievement.  They  are  also  suggestive  of  the  complexity  of 
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the  interaction  between  teaching,  learning  and  technology  which  underscores  the 
need  to  treat  investigations  of  the  effects  of  instructional  technology  in  a 
programmatic  manner.  In  addition,  these  results  point  to  the  need  to  replicate 
studies  in  order  to  increase  the  reliability  of  findings.  Replication  should  focus  on 
the  extent  to  which  findings  are  consistent  across  process  and  product  variables 
and  subject  populations. 

Our  fourth  recommendation  is  to  consider  both  process  and  product  factors 
in  any  research  which  involves  an  interaction  between  teaching,  learning  and 
technology.  In  order  to  improve  the  reliability  of  findings  in  these  areas  replication 
is  necessaiy. 

At  present  we  are  continuing  to  examine  the  effects  of  our  interactive  video- 
based  materials  on  social  problem-solving  and  social  skills  assessment.  Our  most 
recent  study,  which  is  currently  under  analysis,  extended  our  application  of  the 
curricular  materials  to  adolescents  who  have  attentional  difficulties.  It  is  our 
intention  to  both  develop  our  line  of  work  and  to  fill  in  some  gaps  through 
extension  and  replication  of  existing  studies.  In  concluding,  a  final  recommendation 
is  in  order.  Simply,  design  good  studies.  This  is  a  goal  for  us  all  given  the 
logistical  constraints  we  operate  under  which  may  force  compromises  in  a  number 
of  critical  areas  (e,g.,  selection  and  number  of  participants,  time  for  implementation 
and  replication).  Many  of  the  existing  criticisms  of  media  comparison  research 
(e.g.,  novelty  and  teacher  effects)  apply  equally  to  all  educational  research  and 
pertain  more  to  experimental  design  issues  than  to  fundamental  conceptual  fiaws  in 
media  comparisons. 

In  summary,  the  purpose  of  this  paper  has  been  to  describe  a  line  of 
research  which  has  sought  to  address  problems  associated  with  the  existing  status 
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of  research  on  instructional  teclinology.  The  paper  identified  a  number  of  problems 
associated  with  the  failure  to  consider  issues  of  construct  and  content  validity  and 
the  interaction  of  teaching  and  learning  in  ^he  research  process.  Examples  from  a 
program  of  research  on  social  learning  through  interactive  video  were  provided  to 
illustrate  the  ways  in  which  those  problems  could  be  addressed. 
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The  Process  of  Implementing  the  Use  of  Computers  in  Teaching  Secondary  School 
Mathematics  and  the  Effect  on  Student  Learning:  A  Field  Study 
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Stevens  Institute  of  Technology  and 
Lori  Morris,  Educational  Testing  Service 


Abstract 

A  three  year  project  involving  the  collaboration  of  Stevens  Institute  faculty  with  30  mathematics 
teachers  at  5  New  Jersey  high  schools  resulted  in  the  implementation  of  computer  use  in  the  teaching 
of  mathematics.  During  the  academic  year  1990-91,  two  before-and-after,  with-and- without  studies 
of  student  performance  on  in-class  tests  investigated  the  effect  on  student  learning.  Each  student  in 
the  two  samples  of  108  and  71  students  were  given  a  pre-test,  a  post-test,  and  a  delayed  post-test  (to 
measure  retention).  The  analysis  was  performed  on  the  improvement  of  post-test  scores  over  pre- 
test scores,  and  on  the  improvement  of  delayed-post-test  scores  over  pre-test  scores.  Each  student  in 
the  sample  of  71  students  participated  both  in  the  treatment  condition,  using  computers,  as  well  as  in 
the  control  condition,  not  using  computers,  on  different  topics;  these  students  showed  a  statistically 
insignificant  7%  better  performance  on  the  in  the  treatment  condition  over  the  control  condition  on 
the  post-test  improvement  and  a  statistically  significant  18%  better  performance  on  the  delayed-post- 
test  improvement.  The  sample  of  108  students  were  split  between  the  treatment  condition  (41 
students)  and  the  control  condition  (61  students);  the  students  who  studied  with  the  use  of  computers 
showed  a  45%  better  performance  on  the  post-test  improvement  and  a  70%  better  performance  on 
the  delayed-post-test  improvement  over  the  students  who  studied  without  the  use  of  computers.  For 
the  latter  sample,  these  results  represent  almost  a  full  standard  deviation  better  performance.  Data 
showing  the  relationship  between  performance  and  other  factors,  such  as  subject  matter,  school, 
treatment  order,  and  extent  of  computer  use  are  also  presented. 


1.  Introduction 

During  a  three  year  period  from  1988  through  1991,  faculty  and  staff  from  the  Center  for  Improved 
Engineering  and  Science  Education  (CIESE)  at  Stevens  conducted  a  project  at  five  New  Jersey  high 
schools  intended  to  explore  the  effects  of  the  introduction  and  use  of  computers  in  high  school 
mathematics. 

Evaluation  of  this  project  focused  on  three  aspects:  student  performance  (conducted  by  Jurkat,  Skov, 
and  McGinley  from  CIESE);  teacher  knowledge,  activities,  attitudes  and  beliefs  (conducted  by 
Morris  (1992)  from  Educational  Testing  Service);  and  the  administration  and  conduct  of  the  project 
by  Brunner  and  Henriquez  (1991)  from  Bank  Street  College  of  Education.  In  addition,  a  detailed 
case  study  of  the  activities  and  effects  of  the  project  was  done  at  one  of  the  schools  [Allum  (1991)]. 
This  paper  will  concentrate  on  student  performance  results,  summaries  of  the  other  studies  are  in- 
cluded in  Appendices  I,  II,  and  III. 
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2.  Summar>'  of  Results 

Student  performance  was  evaluated  using  a  combination  of  before-and-after,  with-and-without 
computer  use  design.  Two  separate  samples  of  students  were  used. 

In  one  sample  of  108  students,  the  students  in  classes  where  computers  were  used  showed  45% 
better  performance  improvement  (on  post-test  minus  pre-test  scores)  over  students  in  classes  where 
computers  were  not  used,  and  70%  better  retention  (measured  by  delayed-post-test  minus  pre-test 
scores).  These  represent  approximately  1  standard  deviations  in  both  cases. 

In  another  sample,  each  of  71  students  were  taught  and  tested  on  a  topic  where  computers  were  used 
and  on  a  different  topic  where  computers  were  not  used.  The  same  teacher  taught  both  topics  in 
both  conditions.  Overall  in  this  sample,  the  performance  of  students  on  topics  where  computers 
were  used  was  a  statistically  insignificant  7%  better  than  the  performance  of  the  same  students  on 
topics  where  computers  were  not  used,  but  retention  was  18%  better.  Performance  showed  a  wide 
range  of  results  which  varied  with  experimental  conditions,  from  a  67%  reduction  I:  performance 
(in  a  small  class  of  11  students)  to  a  140%  increase  in  performance  (in  another  small  class  of  10 
students).  Both  of  these  extremes  occurred  in  the  same  school. 


3.  Background 

The  standard  approach  to  testing  the  results  of  an  educational  innovation  consists  of  testing  student 
performance  before  and  after  the  introduction  of  the  innovation  and  comparing  those  results  to  a 
control  group  taught  without  the  introduction  of  the  innovation. 

Many  such  studies  have  evaluated  the  use  of  computers  as  the  innovation  being  tested,  and  several 
authors  have  evaluated  the  studies  themselves.  J. A.  Kulik  and  associates  in  a  series  of  reviews 
[Kulik  and  Bangert-Drowns  (1983),  Kulik,  Bangert,  and  Williams  (1983),  Bangert-Drowns,  Kulik, 
and  Kulik  (1985),  Kulik  and  Kulik  (1989)]  found  the  use  of  computers  to  be  generally  effective  in 
improving  performance.  In  their  most  recent  report  [Kulik  and  Kulik  (1989)],  which  reviewed  91 
studies  in  mathematics,  they  concluded  that  the  typical  student  who  learned  or  was  taught  with  the 
use  of  computers  would  outperform  60%  of  the  students  who  learned  or  were  taught  without  the  use 
of  computers. 

In  a  more  recent  paper,  J.  Phillip  Bennett  (August,  1991)  reviewed  the  reviews  of  Kulik  and 
associates  mentioned  above  as  well  as  the  reviews  of  M.D.  Roblyer  and  associates  [Roblyer  (1988), 
Roblyer,  Castine,  and  King  (1988)],  and  H.J.  Becker  (1991).  From  these  he  concluded  that  in 
mathematics  the  use  of  computers  increased  achievement  when  used  as  a  supplement  to  other  forms 
of  teaching.  More  generally,  Bennett  also  concluded  that  the  use  of  computers  improved  students' 
attitudes  toward  computers  and  instruction  but  not  necessarily  toward  mathematics,  that  students 
learn  more  quickly  when  computers  are  used  to  supplement  other  instruction,  and  that  computers 
have  a  greater  effect  on  the  learning  of  disadvantaged  and  low  ability  students  than  the  average  stu- 
dent. 

Several  reasons  have  been  mentioned  as  to  why  the  use  of  computers  might  help  student  learning. 
Some  specifically  related  to  the  operational  aspects  of  computer  use  have  been  discussed  by  Kaput 
(1991)  and  include  the  ability  of  the  computer  to  support  the  visualization  of  intermediate  steps  in 
mathematical  processes,  to  implement  constraint-support  structures  on  both  mathematical  objects  and 
actions  in  microworlds,  to  help  overcome  cognitive  overload  due  to  keeping  connections  between 
structures  and  actions  in  two  or  more  representations  of  mathematical  ideas  {e.g.,  algebraic 
equations  of  functions,  their  graphs,  and  a  simulation  of  the  physics  of  the  situation  being  modeled), 
and  the  general  ability  of  computers  to  make  explicit  such  multiple  linked  representations. 
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On  a  more  situational  and  social/classroom  organizational  level,  the  use  of  computers  makes  it 
possible,  and  maybe  easy,  to  support  teachers  in  forms  of  instruction  and  student  learning  other  than 
the  traditional  lecture-problem -test  paradigm.  Although  not  explicitly  part  of  our  research,  we  have 
observed  examples  of  visual  learning  [as  described  by,  for  example,  Collins,  Brown,  and  Holum 
(1991)3,  constructivism  [as  described  by,  for  example,  Cobb,  Yackel,  and  Wood  (1992)],  and 
collaborative  learning  [as  discussed,  for  instance,  in  Davidson  and  Kroll  (1991)]. 

The  seemingly  simple  approach  of  comparing  student  performance  after  learning  or  being  taught 
"with"  vs.  "without"  is  not  without  its  problems.  The  most  intractable  is  that  it  is  impossible  to 
make  the  students  and  situation  faced  by  those  students  in  the  experimental  group  truly  comparable 
to  the  students  and  the  situation  faced  by  the  students  in  the  control  group.  Different  groups  of 
students  are  used  because  little  insight  can  be  gained  when  the  same  student  is  taught  the  same  mate- 
rial twice,  once  "with"  the  use  of  computers  and  once  "without".  In  addition,  if  alternative  teaching 
and  learning  paradigms  are  used,  maybe  even  without  meaning  to,  when  computers  are  introduced 
into  the  learning  situation,  it  is  difficult  to  separate  the  effects  of  the  alternative  paradigms  from  the 
use  of  computers  themselves. 

In  another  review  of  Kulik  and  his  associates'  work,  Clark  (1985)  concluded  that  there  were  many 
differences  other  than  the  use  of  computers  in  the  groups  of  the  studies  examined  by  Kulik  et.al. 
Betty  Collis  (August/September  1988)  lists  student  cognitive  and  social  characteristics,  teacher 
characteristics,  overall  instructional  strategy,  social  organizations  of  classes  and  classrooms,  social 
and  physical  organization  of  the  computer  work,  and  the  characteristics  of  the  hardware  and  software 
as  variables  that  will  confound  the  results  of  with  and  without  studies. 

It  may  be  concluded  from  this  brief  review  that  "with"  and  "without"  studies  are  valuable,  but  care 
needs  to  be  taken  so  that  the  effect  of  the  innovation  can  be  isolated  as  much  as  possible.  They 
should  be  supplemented  by  evaluations  that  focus  on  the  process  of  the  innovation  itself  and  the 
educational  and  social  situations  in  which  the  teachers  and  learners  find  themselves.  In  addition, 
student  and  teacher  knowledge,  beliefs,  attitudes  and  behavior  before,  during,  and  after  the 
introduction  of  the  innovation  should  also  be  studied. 

The  evaluation  of  the  CIESE  project,  attempting  to  take  these  concerns  into  account,  was  organized 
into  the  three  activities  mentioned  above  at  the  beginning  of  the  paper.  We  did  not  attempt  to 
separate  the  effects  of  the  introduction  of  computers  frcm  any  associated  use  of  teaching  and 
learning  methods  not  present  before  the  introduction  of  computers.  For  this  evaluation,  the  use  of 
computers  includes  the  physical  use  of  the  machines  as  well  as  any  new  or  alternative  teaching  and 
learning  paradigms. 


4.  Project  Rationale 

The  mathematical  skill  and  knowledge  of  students  in  the  United  States  at  the  beginning  of  this 
project,  and  still  now,  leaves  much  to  be  desired.  Lapointe,  Mead,  and  Phillips  (1989)  reported 
that,  on  the  average,  U.S.  students  are  achieving  at  a  substantially  lower  level  of  mathematics 
proficiency  compared  to  students  in  many  other  industrialized  countries.  In  a  recent  study  by  the 
New  Jersey  Basic  Skills  Council  [as  reported  by  Braun  (1992)],  46%  of  new  college  students  fell 
into  the  "lack  proficiency"  category  in  computational  skills,  and  that  only  30%  of  all  new  college 
students  are  completely  proficient  in  basic  math  skills.  It  was  reported  that  mathematics  proficiency 
had  actually  decreased  in  the  most  recent  academic  year. 

There  are  compelling  reasons  to  think  that  using  computers  in  mathematics  education  will  help 
students  learn  more  and  perform  better.  The  studies  cited  above  by  Kulik  cr.al.  and  Bennett  support 
this.  In  addition,  the  U.S.  Congress  Office  of  Technology  Assessment  (1988)  encourages  greater 
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use  of  computer  technology  in  mathematics  instruction  and  several  studies  [e.g.,  Kaput  (1988)  and 
Rubin  (1988)]  have  substantiated  the  potential  to  improve  mathematics  instruction  through  the  use  of 
computer  technology. 

In  addition  to  the  possibility  of  enhancing  student  knowledge  of  mathematics,  there  are  two  other 
reasons  for  using  computers  to  teach  mathematics:  computers  are  considered  a  valuable  artifact  of 
our  society,  lending  seriousness  (sometimes  spuriously)  to  any  activity  involving  them,  and  they  are 
personally  engaging  for  many  people,  particularly  youngsters  [Lepper  (1990)].  Both  of  these  at- 
titudes seem  to  enhance  motivation  and  hence  attention,  which  is  generally  considered  a  necessary 
prerequisite  for  learning. 

Finally,  computers  are  underutilized  in  mathematics  education.  Collis  (1988)  reports  that  80%  of 
the  students  had  never  used  a  computer  in  the  context  of  their  mathematics,  science,  or  English  in- 
struction. Kenneth  Goldberg  (1990),  in  an  update  of  earlier  work,  reported  that  among  480 
mathematics  teachers  in  New  York  State,  only  28%  had  a  computer  in  their  classroom  and  only  54% 
used  computers  at  all  to  supplement  their  teaching,  Becker  (1991)  reported  that  less  than  half  of  all 
secondary  school  mathematics  teachers  had  ever  used  a  computer  at  least  once  in  teaching  and  that 
few  teachers  used  computers  systematically. 

This  underutilization  of  computers  was  observed  by  the  authors'  in  the  five  high  schools  of  the 
project.  Even  though  a  considerable  number  of  the  30  teachers  involved  in  the  study  taught 
programming,  none  of  them  used  computers  to  teach  math  prior  to  the  project. 


5,  Project  Description 

The  project  has  been  described  in  detail  earlier  [Jurkat  eLaL  (1990)].  It  concentrated  on  30  9th"12th 
grade  mathematics  teachers  at  five  New  Jersey  high  schools,  and  involved  up  to  five  CIESE  faculty 
and  staff  and  more  than  a  dozen  school  systems  administrators.  Some  characteristics  of  the  school 
systems  are  given  in  Tabie  1 , 


Table  1,  Participating  High  Schools  and  District  Characteristics 
School: 

I  II  III  IV  V 

District  Population                 19,000  38,800  42.000  320,000  15,000 

School  Population                    1329  1664  1236  1438  842 

Average  Combined  SAT             778  914  700  634  1026 

College  Bound                        26%  84%  55%  48%  91% 

White                                  4 1  %  73  %  1 8  %  2  %  75  % 

Black                                  49  %  20  %  9  %  93  %  1  % 

Hispanic                                 1 0  %  4  %  67  %  3  %  1  % 

Asian                                      -  3%  6%  2%  23% 


Schools  II  and  V  are  in  affluent  suburbs.  School  I  is  in  a  small  city  with  high  unemployment 
surrounded  by  farms  and  poor  rural  areas,  and  Schools  III  and  IV  are  in  disadvantaged  urban  areas. 
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Activities  of  the  project  included: 

-  making  teachers  aware  of  the  potential  of  computers  in  mathematics  education  -  at  monthly 

meetings,  the  teachers  with  the  CIESE  staff  and  faculty  examined  the  mathematics  curricula  at 
the  various  high  schools  to  seek  appropriate  opportunities  for  computer  use,  and  CIESE  people 
acquired  and  demonstrated  appropriate  software  and  suggested  alternative  presentation 
techniques  and  student  activities; 

-  giving  teachers  time  and  the  opportunities  to  examine  and  become  familiar  with  the  hardware  and 

software  useful  for  mathematics  education  -  CIESE  organized  summer  workshops  allowing 
teachers  ample  time  for  individual  and  group  exploration  and  practice  with  the  more  than  250 
software  programs  in  the  CIESE  library';  during  the  academic  year  teachers  could  borrow  the 
softwai'e  for  exploration  with  their  regular  classes  when  department  heads  and  administrators 
could  not  buy  the  software  until  the  teachers  were  sure  they  were  going  to  use  it;  and 

-  interceding  with  school  boards  and  administrators  to  support  the  teachers'  exploration  and 

implementation  of  computer  use  in  mathematics  teaching  -  CIESE  personnel  were  instrumental 
in  persuading  school  boards  and  administrators  to  make  hardware  and  software  available  to 
teachers  and  acquiring  outside  grants  for  hardware  and  software;  CIESE  staff  gained  adminis- 
trative commitment  for  teachers'  stipends  for  workshops  and  substitutes'  availability  for  project 
meetings;  and  the  Stevens  President  and  CIESE  Director  made  presentations  to  school  boards 
and  PTA's  to  have  this  support  maintained  throughout  the  project  duration. 

During  the  first  year  of  the  project  teachers  became  aware  of  the  possibilities  of  using  computers  in 
their  matliematics  classes.  During  the  second  year  they  explored  such  use  in  regular  classes.  During 
the  third  year  they  implemented  this  use  in  many  modules  of  the  entire  curriculum.  Throughout  the 
project,  school  administrators  established  practices  and  budgets  that  would  help  institutionalize  the 
use  of  computers  beyond  the  time  of  the  project. 

The  data  gathering  for  the  student  performance  evaluation  study  occurred  during  the  second  and 
third  marking  periods  of  the  third  year  of  the  project.  Teachers  were  asked  to  participate  on  a 
voluntary  basis;  seven  teachers  did  so. 

The  teachers  were  asked  to  choose  a  topic  for  which  they  either  had  used  computers  or  were  going 
to  use  them.  For  that  topic  they  were  asked  to  give  a  pre-test  before  they  started  teaching  it,  a  post- 
test  at  the  end  of  the  unit,  and  a  delayed-post-test  some  weeks  later  after  other  topics  had  been 
covered.  The  data  gathering  was  to  be  as  unintrusive  as  possible;  teachers  were  asked  to  follow 
their  regular  or  intended  teaching  practices.  The  teachers  designed  each  test  as  if  they  were  giving  it 
for  their  own  grading,  the  pre-test  could  be  given  as  part  of  the  unit  test  on  the  previous  unit,  the 
post-test  was  to  be  their  regular  end-of-unit  exam,  and  the  delayed-post-test  could  be  part  of  their 
marking  period  exam.  Teachers  were  asked  to  follow  their  regular  practice  in  regard  to  announcing 
the  tests  or  giving  them  as  pop-quizzes.  As  it  happened,  only  one  of  the  teachers  was  in  the  practice 
of  giving  a  pre-test  so  students  "knew  something  was  up".  Several  of  the  teachers  concluded  that 
pre-tests  were  a  good  idea  and  are  continuing  to  give  them.  All  tests  were  to  be  graded  on  a  scale  of 
0  to  100%.  Time  between  post-test  and  delayed-post-test,  and  the  number  of  intervening  topics, 
were  not  controlled  across  schools. 

No  attempt  was  made  to  regiment  the  teachers  into  similar  practices,  approaches,  or  styles  when 
using  computers.  The  entire  thrust  of  the  CIESE  Project  was  to  empower  the  teachers  to  develop 
effective  styles  with  which  they  were  individually  comfortable.  This  evaluation  did  not  try  to 
eliminate  the  diversity  in  search  of  research  results  based  on  commonality. 

The  software  used  in  the  sections  taught  with  the  use  of  computers  is  listed  in  Appendix  IV. 
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6.  Project  Results 


6. 1  Sample  1 

For  this  sample  of  71  students,  a  teacher  who  is  teaching  two  sections  (say,  A  and  B)  of  the  same 
course,  taught  each  section  of  the  course  as  follows: 

Topic  I         Topic  II 

Section  A  with  without 

Section  B  without  with 

This  arrangement,  with  enrollment  in  both  classes  kept  constant  from  topic  I  to  topic  II,  guarantees 
that  the  experimental  and  control  groups  consist  of  exactly  the  same  students  aod  teacher,  with  only 
the  topic  and  the  order  of  computer  use  differing.  This  design  is  an  extension  of  a  design  recently 
used  by  Ganguli  (1990)- 
Unfortunately,  only  one  teacher  (at  School  I)  with  three  sections  was  able  to  meet  all  these 
conditions  during  the  time  of  the  study.  By  allowing  the  teacher  to  be  different  in  the  two  sections, 
three  other  sections,  one  at  School  I  and  two  at  School  II,  were  able  to  be  included. 

The  teacher  at  School  I  taught  Congruence  and  Similar  Triangles  and  Angles  Related  to  Circles  in 
Geometry  1,  Geometry  2,  and  Geometry  Honors,  Two  teachers  at  School  II  taught  Geometric 
Vocabulary,  Angle  Measures,  Parallel  Lines,  Area,  Perimeter,  Similar  Triangles,  and  Polygons  in 
various  combinations  to  two  sections  of  Math  9. 

The  dependent  variables  for  Sample  1  are: 

PRE_W     =  student  score  (0-100%)  on  pre-test  for  topic  taught  with  the  use  of  computers 
POST_W     =  student  score  (0-100%)  on  post-test  for  topic  taught  with  the  use  of  computers 
DEL_W    =  student  score  (0-100%)  on  delayed-post-test  for  topic  taught  with  the  use  of 
computers 

POSDIFW  =  performance  improvement  with  the  use  of  computers  (POST_W  minus  PRE_W) 
DELDIFW    =  retention  improvement  with  the  use  of  computers  (DEL_W  minus  PRE_W) 

PRE__WO    =  student  score  (0-100%)  on  pre-test  for  topic  taught  without  the  use  of  computers 
POST_WO    =  student  scof;  (0-100%)  on  post-test  for  topic  taught  without  the  use  of 
computers 

DEL_WO    =  student  score  (0-100%)  on  delayed-post-test  for  topic  taught  without  the  use  of 
computers 

POSDIFWO    =  performance  improvement  without  the  use  of  computers  (POST_WO  minus 
PRE^WO) 

DELDIFWO    =  retention  improvement  without  the  use  of  computers  (DEL_WO  minus 
PRE^WO) 

Independent  variables  included: 

SCHOOL    =  school  student  attended  (I  and  II  above) 

COURSE    =  one  of  Geometry  Honors,  Geometry  I,  Geometry  II,  and  9th  Grade  Math 
TOTDAYS     =  nuniber  of  instructional  periods  during  which  some  use  of  the  computer  was 
made  during  the  time  the  topic  was  taught  with  computers 
SEX     =  sex  of  student 
GRADE    =  grade  level  of  student  (9  through  12) 
GRADPMRK     =  student  grade  in  math  course  in  the  previous  marking  period  (0-100%) 

ORDER     =  order  in  which  use  of  computers  was  made  in  sections  that  participated  in  both 
"with"  and  "without"  conditions  (1  =  computer  first,  2  =  computer  second) 
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Student  performance  as  related  to  levels  of  these  independent  variables  is  given  Tables  2  through  9. 
Table  2>  Overall  variation  with  and  without  Computer  Use  for  Sample  1 


Students        Mean  S.D. 
POSDIFW  71  32.0  20.2 

POSDiFWO  71  29.9  14.1 

t  =  .94         p  =  .352 

DELDIFW  71  27.5  15.2 

DELDIFWO  71  23.3  15.3 

t  =  2.00        p  =  .049  * 


One  star  (*)  denotes  statistical  significance  at  the  .05  level  and  two  stars  (**)  denotes  statistical 
significance  at  the  .01  level.  In  subsequent  tables,  the  significance  level  is  calculated  from  statistical 
tests  conducted  on  the  percentage  scores  transformed  with  the  arcsine  transform:  y  = 
2arcsin(sqrt(x)).  This  is  done  to  regain  the  statistical  independence  of  the  mean  and  variance  which 
is  lost  vAitn  the  data  are  ratios  and  percentages. 

Table  2  shows  that  the  student  performance  on  the  post-test  minus  the  pre-test  is  a  statistically 
insignificant  7%  better  in  the  "with"  condition  while  student  retention  (delayed-post-test  minus  pre- 
test) is  18%  better  with  the  "with"  condition,  significant  at  the  5%  level. 

Table  3.  Variation  With  Days  of  Computer  Use  During  Unit 

Computers  Used  For  Part  or  All  of: 


6 

10 

19 

Signif 

Performance 

Periods 

Periods 

Periods 

Level 

Measure 

(0-100%) 

(0-100%) 

(0-100%) 

(F  ratio) 

Number  of  Students 

11 

46 

14 

GRADPMRK 

69.0 

75.4 

72.9 

2.34 

PRE  W 

25.0 

35.9 

40.1 

2.48 

POST  W 

34.3 

73.8 

70.5 

20.74 

DEL  W 

38.6 

67.3 

65.7 

11.14 

POSDIFW 

9.3 

37.9 

30.4 

8.81 

DELDIFW 

13.6 

31.4 

25.6 

5.18 

Table  3  shows  that  in  topics  during  which  the  computer  was  used  for  part  or  all  of  10  and  19 
periods,  significant  improvement  in  performance,  as  seen  in  the  post-test  improvement  (POSDIFW) 
and  the  delayed-post-test  improvement  (DELDIFW),  was  exhibited  by  the  students.  More  details  on 
this  effect  will  be  shown  by  the  interaction  analysis  in  Tables  8  and  9  below. 

Table  4  shows  that  student  gain  on  unit  test  scores  (POST_W  and  POST^WO)  and  retention  test 
scores  (DEL_^W  and  DEL^WO)  over  pre-test  scores  is  about  the  same,  even  though  the  School  I 
scores  were  significantly  higher  than  the  School  II  scores  on  the  raw  test  scores.  Also,  there  is  no 
effect  of  school  on  improvement  "with**  or  "without"  or  retention  ''with"  or  "without". 
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Table  4.  Variation  with  School 


School  I 

School  II 

Signif 

Perform  q  n 

A  VP  5?cnre 

Level 

CO-100%) 

CP  ratio"! 

Number  of  students 

50 

21 

lA  1 

I'^i  A 

001 

JrKti  vV 

AO  fi 

jL  /  •  I 

33  10 

«/  i  tXJ 

17  55 

** 

46  0 

30  44 

** 

POSDIFW 

32.8 

29.9 

.060 

DELDIFW 

28.9 

24.2 

.012 

PRE  WO 

38.7 

25.0 

15.90 

POST  WO 

68.3 

55.8 

10.20 

** 

DEL  W 

61.9 

48.7 

9.62 

** 

POSDIFWO 

29.6 

30.8 

2.01 

DELDIFWO 

23.2 

23.7 

1.04 

Table  5>  Variation  with  Course 


Geometry 

Geometry 

Geometry 

Math 

Signif 

Performance 

Honors 

Geom  1 

Geom  11 

Math  9 

l^vel 

Measure 

(0-100%) 

(0-100%) 

(0-100%) 

(0-100%) 

(F  ratio) 

N  of  students 

15 

21 

14 

21 

GRADPMRK 

79.7 

71.0 

72.9 

73.4 

2.94 

» 

PRE  W 

52.5 

32.5 

40.1 

21.8 

17.50 

** 

POST  W 

76.8 

73.0 

70.5 

51.6 

6.28 

DEL  W 

73.3 

69.2 

65.7 

46.0 

11.01 

** 

POSDIFW 

24.3 

40.6 

30.4 

29.9 

.99 

DELDIFW 

20.8 

36.8 

25.6 

24.2 

2.13 

PRE  WO 

54.1 

30.7 

34.0 

25.0 

13.87 

** 

POST  WO 

69.3 

67.6 

68.1 

55.8 

2.41 

* 

DEL  WO 

63.0 

61.4 

61.3 

48.7 

3.19 

* 

POSDIFWO 

15.1 

36.9 

34.1 

30.8 

7.46 

** 

DELDIFWO 

8.9 

30.7 

27.3 

23.7 

6.25 

GRADPMRK  is  used  as  a  proxy  for  the  basic  level  of  performance  of  the  students  in  each  section. 
As  might  be  expected,  the  Geometry  Honors  students  performed  better  in  the  previous  marking 
period,  on  the  average,  than  the  students  in  the  other  classes.  There  were  no  significant  effects  on 
performance  or  retention  improvement  for  the  topics  for  which  computers  were  used.  However, 
there  were  significant  effects  of  topic  on  performance  and  retention  improvement  for  those  topics 
taught  without  the  use  of  computers.  The  lower  level  students  (those  in  Geometry  I  and  II  and  in 
Math  9)  improved  significantly  more  than  the  Honors  students. 

It  is  interesting  to  note  that  although  there  is  a  statistically  significant  difference  in  the  average  test 
scores  of  the  classes  in  the  section  taught  with  the  use  of  computers,  these  differences  are  no  longer 


evident  in  the  POSDIFW  and  DELDIFW.  These  differences  are  small  in  the  Honors  class,  which 
can  be  attributed  to  the  high  pretest  scores,  PRE_W  and  PRE_WO,  which  does  not  allow  for  much 
improvement.  These  Honors  students  knew  quite  a  bit  of  geometry  before  they  began  the  topic. 


Table  6.  Variation  with  Sex  of  Student 

Signif 

X  \yl  Iv/i  I  I1CU1VV> 

Female 

Male 

T  ^vpI 

Measure 

(0-100%) 

(0-100%) 

(F  ratio) 

Number  of  Students 

37 

23 

GRADPMRK 

76.2 

72.4 

2.19 

PRE  W 

41.1 

30.1 

9.54  ** 

POST  W 

74.1 

71.1 

.56 

DEL  W 

70.0 

61.8 

3.58 

POSDIFW 

33.1 

41.0 

4.78  * 

DELDIFW 

28.9 

31.7 

1.84 

PRE  WO 

40.1 

32.8 

.05 

POST  WO 

67.7 

67.4 

.91 

DEL  WO 

59.7 

58.1 

.54 

POSDIFWO 

27.6 

34.6 

.03  * 

DELDIFWO 

19.7 

25.3 

.15 

The  only  statistically  significant  differences  shown  in  Table  6  indicate  that  the  females  had  a  higher 
average  score  on  the  pretest,  PRE^W,  in  those  topics  during  which  the  computer  was  used,  but  that 
the  males  improved  significantly  more,  POSDIFW.  However,  this  difference  in  improvement 
between  the  sexes  was  no  longer  evident  in  the  delayed  post  lest,  DELDIFW. 

Table  7  shows  that  grade  level,  which  also  indicates  age,  had  no  statistically  significant  relationship 
with  performance. 

For  sample  1  there  was  a  significant  three-way  interaction  between  SCHOOL,  ORDER,  and 
computer  use  ("with"  vs.  "without")  on  both  performance  and  retention  improvement,  significant 
two-way  interactions  between  ORDER  and  computer  use  ("with"  vs  "without")  on  both  performance 
and  retention  improvement,  and  a  significant  main  effect  of  ORDER  on  performance  improvement. 
The  performance  improvement  interactions  are  shown  in  Table  8  and  the  retention  improvement 
interactions  are  shown  in  Table  9. 

It  can  be  seen,  particularly  from  Table  8.2,  that  greater  student  performance  improvement  is  gained 
"with"  the  computer  than  "without";  the  effect  is  mainly  due  to  the  16.5%  improvement  "with" 
compared  to  "without"  in  order  1:  with  computer  first. 
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Table  7.  Variation  with  Grade  Level 


Q  th 
y  111 

10  th              11  th 

I              111                                                        JL    X  111 

12  th 

JL  ^  III 

SlPTIlf 

W7  1^1  111 

X^d  iVJX  lllcll  IL'C 

Grade 

Level 

CF  ratio) 

N  of  students 

28 

25  4 

3 

77  Q 

72  4              67  5 

75  0 

2  98 

PRF  W 

38  7 

10 

POST  W 

74. 

71  1               71  5 
/i«i                1 1 

75  7 

31 

HFT  W 

LyCL^  YV 

fn  ()           67  0 

79  0 

07 

POSDIFW 

36.8 

34.8  38.5 

37.0 

.53 

DELDIFW 

28.4 

30.8  34.0 

33.3 

.08 

PRE  WO 

42.3 

33.5  27.5 

35.3 

.86 

POST  WO 

67.4 

67.0  67.5 

74.0 

.26 

DEL  WO 

56.8 

60.3  61.5 

67.3 

.37 

POSDIFWO 

25.2 

33.4  40.0 

38.7 

1.20 

DELDIFWO 

14.6 

26.8  34.0 

32.0 

2.84 

Table  8:  Post-test  Improvement  Interactions  for  Students  who  were  Taught  both  With  and  Without 

the  Use  of  Computer 

(The  number  of  students  in  each  cell  is  given  in  parentheses  after  the  post-test  improvement  average 

for  that  cell 

8.1  POSDIFW  vs.  POSDIFWO  bv  ORDER  and  SCHOOL: 

School  1  School  II 

(F  =  19.89,  p  =  .000)  with     without       with  without 

order  1:  with  computer  first  30(14)  34  (14)      53  (10)  34  (10) 

order  2:  with  computer  second  34  (36)  28  (36)        9  (1 1)  28  (1 1) 

8.2  POSDIFW  vs.  POSDIFWO  by  ORDER: 

(F  =  7.30,  p  =  ,009)  with   _  without_ 

order  1:  with  computer  first  39.6  (24)  34.0  (24) 

order  2:  with  computer  second  28. 1  (47)  27.9  (47) 

8.3  POSDIFW  and  POSDIFWO  average  bv  ORDER:      (F  =  16.06,  p  =  .000) 
order  1:  with  computer  first  36.8  (24) 

order  2:  with  computer  second  28.0  (47) 


Both  Tables  8  and  9  show  that  there  is  some  tendency  for  the  students  in  sections  where  the 
computer  was  used  first  to  perform  better  on  both  the  unit  tests  and  retention  tests.  This  order  effect 
was  not  expected,  nor  are  we  aware  that  it  has  been  mentioned  in  the  literature.  A  partial 
explanation  might  be  that  the  teachers  may  have  preferred  to  use  the  computer  in  the  stronger 
sections  first. 
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Table  9:  Delayed  Post-test  Improvement  Interactions  for  Students  who  were  Taught  both  With  and 

Without  the  Use  of  Computers 
(The  number  of  students  in  each  cell  is  given  in  parentheses  after  the  post-test  improvement  average 

for  that  cell) 

9.1  DELPIFW  vs.  DELDTFWO  bv  ORDER  and  SCHOOL: 

School  I  School  II 

(F  =  32,24,  p  =  .000)        with     without         with  without 
order  1:  with  computer  first  26(14)  27  (14)         36(10)  15  (10) 

order  2:  with  Computer  second  30  (36)  22  (36)         14  (1 1)  32  (11) 

9.2  DELDIFW  vs.  DELDTFWO  by  ORDER: 

(F  =  1L27,  p  =  ,001)  with  without 

order  1:  with  computer  first  30  (24)  22  (24) 

order  2:  with  computer  second  26  (47)  24  (47) 


Sample  2 

Not  all  of  the  30  teachers  in  the  CIESE  Project  taught  2  sections  of  the  same  course,  nor  were  able 
to  be  paired  with  other  teachers  under  Sample  1  conditions.  Those  teachers  that  did  not,  but  still 
wanted  to  participate  in  the  study,  were  "paired"  with  others  teaching  the  same  topic  to  students  in 
the  same  grade,  but  possibly  to  different  students.  This  sample  contained  108  students  whose  per- 
formance is  shown  in  the  following  tables.  Sixty-one  (61)  of  these  students  were  in  sections  using 
computers  and  forty-seven  (47)  were  in  sections  not  using  computers. 

The  dependent  variables  in  this  sample  have  slightly  different  meanings;  they  were  given  different 
names  to  remind  readers  (and  the  investigators!)  of  the  distinction.  In  Sample  1  each  student  took  a 
pre-test,  post-test,  and  delayed-post-test  for  both  "with"  and  "without"  conditions,  whereas  in  the 
current  sample,  Sample  2,  each  student  took  the  tests  for  only  one  of  the  "with"  or  "without"  con- 
ditions. The  dependent  variables  for  Sample  2  are: 

PRETEST  =  student  score  (0-100%)  on  pre-test 

POSTTEST  =  student  score  (0-100%)  on  post-test 

DELTF^T  =  student  score  (0-100%)  on  delayed-post-test 

POST_PRE  =  performance  improvement  (POSTTEST  minus  PRETEST) 

DEL^PRE  =  retention  improvement  (DELTEST  minus  PRETEST) 

Table  10  shows  that  post-test  improvement  with  the  computer  was  about  45%  greater  than  the 
improvement  without  the  computer,  and  the  delayed-post-test  improvement  with  the  computer  was 
70%  greater  than  the  improvement  without  the  computer.  These  difference  represents  about  1 
pooled  standard  deviation,  which  is  at  the  high  end  of  the  range  of  such  results  reported  by  various 
studies  reviewed  in  Roblyer,  Castine,  and  King  (1988).  This  latter  result  may  be  interpreted  as 
saying  that  the  median  student  under  the  "with"  condition  performed  at  a  level  that  exceeded  83%  of 
the  students  under  the  "without"  condition. 
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Table  10.  Performance  and  Retention  Improvement  for  Students  Who  Learned  or  were  Taught  either 

With  or  Without  the  Use  of  Computers 


POST_PRE  "with"  (TOTDAYS  >  0) 
POST_PRE  "without"  (TOTDAYS  =  0) 
difference 
pooled  s.d. 

t-value:  5.08  df:  106         2-tail  P:  .000 

DEL_PRE  "with"  (TOTDAYS  >  0)  47             66.3  23.1 

DEL_PRE  "without"  (TOTDAYS  =  0)  ^1  39J  2M 

difference  27.8 

pooled  s.d.  26.8 

t-value:  4.83           df:  106  2-tail  P:  .000 


Number 

of  Cases 

Mean 

S.D. 

47 

72.4 

19.1 

61 

49.9 

24.7 

22.5 

22.9 

Signif 

Performance 

Level 

Measure 

No  Use 

3  Periods 

15  Periods 

(F  ratio) 

N  of  students 

61 

27 

20 

GRADPMRK 

79.8 

86.9 

74.0 

9.00 

PRETEST 

32.1 

19.6 

15.0 

5.32 

POSTTEST 

82.0 

92.2 

87.2 

3.92 

DELTEST 

71.6 

91.5 

73.8 

8.06 

POST  PRE 

49.9 

72.6 

72.2 

13.03 

DEL  PRE 

39.5 

71.9 

58.8 

13.96 

Table  11  shows  that  the  impiovement  in  perfonnance,  POST^PRE  and  DEL^PRE,  are  significantly 
higher  for  the  two  levels  of  computer  use  compared  to  no  use.  The  students  in  sections  taught  with 
the  use  of  computers  (3  periods  and  15  periods)  had  significantly  lower  PRETEST  scores,  showing 
they  started  with  less  knowledge  of  the  topic,  and  performed  better  on  the  POSITEST  and 
DELTEST  than  the  students  in  the  No  Use  sections.  Also,  performance  and  retention  was  not 
greater  when  15  periods  were  used  as  compared  to  3  periods. 

No  interaction  effects  between  School  and  Number  of  Periods  computers  were  used  were  calculated 
due  to  empty  cells  in  the  matrix  of  interactions. 

Variation  with  SCHOOL  is  coupled  with  variation  with  COURSE  in  that  the  sections  in  these 
schools  were  from  the  following  courses: 

School  I  General  Math  I 

School  II  Math  1 1 

School  III  Algebra  2 
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Table  12.  Variation  with  School/Course  and  Number  of  Periods  Used 


School  I 

School  II 

School  III 

Signif  Level 

(0-100%) 

(0-100%) 

fO-100%) 

(F  ratios) 

Performance 

No  15 

No  3 

No 

Main  Effects 

Measure 

Use  Pers 

Use  Pers 

Use 

School  Use 

GRADPMRK 

76.7  74.0 

86.2  86.7 

72.7 

10.17  **  .53 

PRETEST 

26.7  15.0 

26.9  19.6 

50.0 

4.62  *  2.19 

POSTTEST 

78.0  87.2 

85.8  92.2 

81.1 

1.62  1,82 

DELTEST 

72.9  73.8 

77.7  91.5 

58.5 

3.53  *  2.26 

POST  PRE 

51.3  72.2 

58.8  72.6 

31.1 

6.61  **  7.34  ** 

DEL  PRE 

46.2  58.8 

50.8  71.9 

8.5 

12.76       6.21  ** 

There  were  no  significant  differences  found  in  the  variation  of  performance  and  retention  with  SEX 
of  student  or  GRADE,  and  there  were  no  significant  interactions  between  SEX  and  TOTDAYS  or 
GRADE  and  TOTDAYS. 


A  regression  was  performed  on  the  performance  and  retention  of  these  108  students,  resulting  in  the 
following  two  equations.  The  numbers  in  parentheses  under  the  coefficients  represent  the  probability 
of  having  the  observed  coefficient  due  to  chance  alone  when  its  true  value  is  actually  zero: 

POSTTEST  =  13.  +   .80*GRADPMRK  +  .20*PRETEST  +  .87*TOTDAVS 

(.0000)  (.0053)  (.0142) 

Multiple  R  =  .587  Standard  Error  =  15.3 


This  equation  shows  that,  after  taking  into  account  how  good  the  student  is  (GRADPMRK)  and  how 
much  the  student  knew  about  the  subject  before  it  was  taught  (PRETEST),  each  period  during  which 
the  computer  was  used  (TOTDAYS)  is  associated  with  a  .87%  point  gain;  ten  periods  of  use  would 
be  related  to  an  8,7%  gain  in  performance, 

DELTEST  =  -39.8  +  1.38*GRADPMRK  +  .10*PRETEST  +  .76 '^TOTDAYS 

(.0000)  (.3354)  (.1392) 

Multiple  R  =  ,628  Standard  Error  =  18.4 


In  this  equation,  neither  of  the  coefficients  of  PRETEST  nor  TOTDAYS  can  be  concluded  to  be 
significantly  different  from  zero;  no  linear  relationship  is  concluded.  This  means  that  the  student's 
grade  in  the  previous  marking  period  (GRADPMRK),  which  is  a  proxy  for  how  good  each  student  is 
in  mathematics,  is  the  only  one  of  these  variables  that  is  related  to  retention  (DELTEST). 


6.2  Related  Studies 

Three  additional  research  studies  on  the  CIESE  project  were  in  progress  at  the  same  time  as  this 
student  performance  analysis.  An  Educational  Testing  Service  study  on  teacher  and  student  attitudes 
was  conducted  by  Lori  Morris  [Morris  (1992)].  A  study  of  overall  school  system  response  and 
interaction  with  the  CIESE  project  was  conducted  by  researchers  from  the  Bank  Street  College  of 
Education  [Brunner  and  Henriquez  (1990)].  The  third  study  was  an  in-depth  analysis  of  the  CIESE 
project  on  the  behavior  and  relationships  among  mathematics  teachers  in  one  of  the  CIESE  high 
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schools.  This  work  was  done  by  Keith  Allum  as  his  Ph.D.  dissertation  for  the  Sociology 
Department  of  Princeton  University  [Allum  (1991)]. 

Excerpts  from  the  writings  of  these  researchers  are  included  with  this  paper  as  Appendices  I,  II,  and 
III.  These  excerpts  are  not  intended  to  provide  definitive  summaries  of  each  research  analysis. 
Rather,  they  are  provided  in  order  to  supply  the  reader  with  a  sense  of  the  context  in  which  this 
student  performance  research  was  conducted. 


7.  Conclusion 

A  premise  of  the  CIESE  project  has  been  that  the  teaching/learning  process  in  secondary 
mathematics  could  be  enhanced  through  judicious  use  of  computer  technology.  This  student  perfor- 
mance field  study  provides  strong  support  for  that  position.  We  have  observed  significant 
enhancements  in  student  performance,  particularly  in  long-term  retention  in  a  study  which  examined 
a  variety  of  computer  integration  strategies  in  a  mix  of  school  settings  and  curricula  applications. 

A  fundamental  aspect  of  this  study,  which  differentiates  it  from  many  others,  is  that  teachers 
involved  had  been  part  of  a  computer-mediated  education  development  project  for  more  than  two 
years.  Other  studies  [Sheingold  and  Hadley  (1990)]  indicate  that  it  takes  teachers  several  years  to 
become  mature  in  their  use  of  computer  technology  in  the  classroom.  The  question  then  arises 
regarding  possible  further  gains  in  student  performance  after  these  teachers  have  had  additional  time 
to  refine  their  computer-mediated  instructional  approach. 

Additional  issues  for  future  research  include  refinements  regarding  the  relative  advantages  of 
computer  laboratories  versus  the  one  computer  classroom,  as  well  as  analyses  of  how  computers 
facilitate  particular  learning  strategies,  such  as  visualization,  constructivism,  and  collaboration. 

Finally,  it  is  important  to  note  that  this  study  of  student  performance  was  carried  out  in  a  context  in 
which  teachers  were  engaged  in  an  in-service  project  which  was  supported  by  their  administrators  as 
well  as  by  their  school  boards  and  local  communities.  These  external  factors  helped  teachers  pursue 
instruction  as  they  wished  and  motivated  their  utilization  of  technology.  The  impact  of  school 
environment  and  the  potential  for  school  restructuring  to  promote  student  performance  through  more 
effective  integration  of  technology  in  instruction  are  emerging  as  key  issues  for  researchers  and 
policy  makers  in  the  years  ahead. 
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9.  Appendix  I:  Teacher  Knowledge,  Activities,  Attitudes  and  Beliefs 

By  the  third  year  of  the  project  all  but  two  of  the  thirty  teachers  who  started  the  project  (one  of 
which  left  the  project  in  the  first  year)  were  using  computers  throughout  the  school  year.  Each  had 
access  to  a  computer  projection  station  to  use  in  their  lectures  and  group  problem  solving  sessions 
and  four  of  the  high  schools  had  computer  laboratories  available  to  which  mathematics  teachers 
could  bring  their  entire  classes  for  individual  student  use  of  computers.  All  had  the  software  they 
wanted  available  to  them. 

This  software  and  hardware  was  acquired  through  school  system  capital  budgeting  and  annual 
operational  funding,  as  well  as  from  grants  and  gifts.  All  department  heads  and  mathematics 
supervisors  were  budgeting  for  software  acquisition  and  hardware  maintenance.  Nearly  all  the 
teachers  expressed  plans  to  continue  using  the  computers  after  CIESE  support  was  curtailed. 

The  following  observations  and  beliefs  were  elicited  from  teachers  and  administrators  during  private 
interviews  with  Dr.  I^ri  Morris  from  Educational  Testing  Service  [Morris  (1992)]  throughout  the 
project's  three  years.  She  adopted  a  stance  apart  from  the  CIESE  staff  and  faculty,  becoming  a 
confidant  of  the  teachers  and  administrators  which  allowed  them  to  comment  on  and  criticize  project 
activities  without  confrontation.  When  appropriate.  Dr.  Morris  made  the  CIESE  staff  aware  of 
complaints  and  dissatisfactions  which  were  then  addressed  by  the  staff. 

The  teachers  generally  indicated  that  they  adopted  different  teaching  styles  when  the  use  of 
computers  was  integrated  into  their  classes  and  laboratories  and  that  the  use  of  computers  had  an 
effect  on  their  teaching  assignments  and  approach  to  their  craft. 

Computer  use  was  not  integi^ated  to  the  same  extent  among  all  of  the  project  teachers.  Some, 
particularly  those  that  had  equipment  and  software  available  early,  began  to  experiment  with 
computer  use  in  the  second  semester  of  the  project  and  may  be  considered  experienced  users  by  the 
end  of  the  third  year.  Others,  both  due  to  their  own  inclination  and/or  lack  of  equipment,  were  not 
able  to  routinely  use  computers  until  the  third  year  of  the  project,  and  should  still  be  considered 
novices  at  the  time  the  evaluation  took  piace.  These  differences  could  account  for  some  of  the 
variability  in  the  student  performance  results  presented  above. 


48 


50 


17 

CIESE  staff  in  general  adopted  a  non-directive  approach  which  was  credited  by  the  teachers  in 
empowering  the  teachers  and  providing  them  with  enough  confidence  to  independently  develop  their 
own  criteria  for  "good"  and  "bad"  software.  The  formal  presentations  to  each  school's  board  of 
education  and  informal  discussions  with  administrators  was  credited  by  those  involved  in  the  project 
with  playing  an  integral  role  in  resolving  the  two  most  critical  issues  of  having  sufficient  time  for 
teachers  and  money  for  hardware,  software,  and  supplies. 

Both  summer  workshops  were  deemed  successful  by  the  teachers  because  time  was  provided  for 
teachers  to  reach  their  goals;  the  workshops  were  flexible  enough  for  teachers  to  informally  work  in 
groups  or  alone;  teachers  were  provided  with  an  opportunity  to  interact  with  other  math  teachers 
outside  of  their  schools,  and  all  teachers  felt  they  were  treated  like  professionals. 

The  advantages  of  the  computer  mentioned  most  frequently  by  teachers  were  that  computers 
enhanced  the  quality  and  quantity  of  visual  examples  (its  speed  allowed  more  examples  in  the  same 
amount  of  time),  enabled  more  material  to  be  covered  in  a  shorter  amount  of  time,  offered  students 
autonomy,  enabled  students  to  make  mistakes  privately  in  front  of  a  computer  rather  than  in  front  of 
the  entire  class,  increased  discussion  of  mathematics  among  students,  and  changed  the  role  of  the 
teacher  from  an  authoritarian  to  a  coach. 

Teachers  of  lower  academic  level  students  noted  that  the  effects  of  the  computer  were  subtle  but 
important,  claiming  that  the  computer  improved  student's  self-esteem  and  attitudes  .award  mathe- 
matics and  lessened  discipline  problems  within  the  group. 

Teachers  of  higher  academic  level  students  tended  to  use  the  computer  to  encourage  higher  order 
thinking  skills  through  the  use  of  the  discovery  method  of  learning. 

Integrating  computers  into  the  curriculum  encouraged  teachers  to  self-reflect  on  their  teaching  styles 
and  pedagogies.  Becoming  involved  with  computers,  learning  about  them,  and  exploring  their  use 
in  teaching  served  to  rejuvenate  many  of  the  teachers. 

The  process  of  integrating  computers  encouraged  collaborative  relations  among  the  teachers,  and  in 
some  cases,  between  administrators  and  teachers.  Project  staff  successfully  estaolished  a  profes- 
sional, collegial  relationship  with  the  teachers, 

Morris,  L,V.  (1992)  Final  Evaluation  Report  of  the  CIESE  Project,  Princeton,  NJ:  Educational 
Testing  Service 


10.  Appendix  H:  Project  Administration  and  Conduct 

This  evaluation  was  carried  out  by  staff  [Brunner  and  Henriquez  (1991)]  of  The  Center  for  Children 
and  Technology  at  Bank  Street  College  of  Education,  Fourteen  administrators  in  all  five  schools 
were  interviewed:  six  of  these  were  department  heads,  two  were  assistant  principals,  two  were 
district-wide  curriculum  coordinators,  and  four  were  district  superintendents. 

Questions,  in  addition  to  the  school's  administrative  structure,  student  body,  and  assessment 
techniques  included  inquiries  about  the  educator's  overall  expectations  of  the  collaboration,  their 
evaluation  of  its  success,  their  definition  of  its  major  challenges,  and  their  future  expectations. 

A  major  finding  was  that  the  regular  school  visits,  technical  support,  training,  summer  workshops 
and  conferences  provided  opportunities  for  teachers  to  meet,  discuss  common  issues  and  expe- 
riences, and  to  solve  common  and  individual  problems.  This  created  the  preconditions  for  positive 
collaboration  between  project  staff  and  school  personnel. 
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The  administrators  generally  felt  that  the  collaboration  between  CIESE  personnel  and  the  high  school 
teachers  was  the  major  reason  for  the  degree  of  success  in  the  integration  of  technology  into  the 
participating  schools.  They  considered  the  training  in  the  use  of  pertinent  software  as  highly  useful, 
and  they  regarded  the  regular  meetings  with  CIESE  staff  as  a  valuable  source  of  support  and  inspi- 
ration to  their  staff  members. 

The  great  majority  of  the  educators  interviewed  consider  technology  integration  critical  to  the 
education  of  their  students. '  Those  that  were  not  entirely  convinced  were  from  the  school  which  was 
the  most  remote  of  the  project  schools,  whose  administrators  were  the  most  traditional,  and  whose 
management  style  was  the  most  top-down  and  hierarchical. 

Most  of  the  district  level  administrators  joined  the  project  when  it  was  already  in  progress;  the 
school  level  administrators  and  teachers  obviously  had  joined  the  project  at  its  beginning.  The 
school  level  administrators  considered  themselves  closely  tied  to  the  project;  they  felt  they  were 
hand-picked  because  they  were  in  agreement  with  the  project's  goals  and  were  in  a  position  to  imple- 
ment them.  In  this  regard,  the  project,  in  some  sense,  was  "doomed  to  succeed".  The  district  level 
admhiistrators,  joining  the  project  when  it  was  moving  toward  success,  had  no  reason  to  distance 
themselves  from  it. 

The  collaboration  seemed  to  have  a  positive  effect  on  the  relationship  between  schools  and  their 
district  offices.  Neither  level  believed  they  "owned"  the  project;  each  thought  the  other  was  impor- 
tant for  its  achievements. 

When  asked  how  the  project  fitted  into  the  administrators'  plans  they  responded  variously  with  the 
hope  that  it  will  improve  the  image  of  the  mathematics  department,  the  expectation  that  there  would 
be  a  positive  change  in  standardized  test  scores,  the  perception  that  the  project  supplemented 
ongoing  efforts  in  technology  integration  or  the  chance  to  get  any  kind  of  technology  into  the  de- 
partment at  all. 

In  addition,  the  expectations  for  the  project  included  a  demonstratable  improvement  in  the  teaching 
skills  of  the  teachers  and  a  positive  change  in  the  teaching  methods  used  in  mathematics  classes. 

Most  administrators  saw  an  improvement  in  teaching  methods  as  technology  was  integrated  into  the 
mathematics  curriculum  and  some  mentioned  the  improved  collegial  relationship  among  their 
teachers. 

A  third  of  the  administrators  were  not  aware  of  any  particular  contribution  their  schools  made  to  the 
integration  of  technology  into  the  mathematics  curriculum. 

The  implications  of  these  perceptions  was  that  the  administrators  saw  the  CIESE  Project  most 
responsible  for  the  improvements  they  mentioned.  A  few  of  the  administrators  complained  about  the 
lack  of  time  and  difficulties  in  involving  other,  non-project  teachers  in  similar  activities. 

Staff  development  was  cited  most  often  as  the  most  rewarding  aspect  of  the  collaboration*.  The 
trainir  ^,  collegial  contacts,  the  introduction  of  computers,  and  access  to  the  CIESE  staff  were  all 
consiaered  rewarding. 

Time  and  money  were  considered  the  major  obstacles  to  the  success  of  the  project  as  a  whole. 
Teachers  needed  more  time  to  learn,  more  time  to  meet,  more  time  to  plan,  more  time  to  digest 
what  they  have  learned,  and  more  time  to  experiment  with  applications  in  the  classroom.  Money 
was  needed  for  more  computers  and  software. 
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Problems  included  jealousy  among  the  teachers  who  were  not  part  of  the  project,  the  unwillingness 
of  some  tochers  to  become  genuinely  involved  in  the  project,  and  the  lack  of  cooperation  from  the 
administrators  of  a  particular  school. 

Most  of  the  educators  thought  there  was  some  lack  of  clarity  about  the  goals  of  the  collaboration. 
This  was  deliberate  on  the  part  of  the  CIESE  staff  since,  at  the  beginning,  it  was  not  evident  that  all 
the  participating  schools  would  buy  into  a  common  set  of  goals  other  than  to  improve  mathematics 
education.  Respondents  talked  about  setting  clearer  goals  at  the  beginning  of  the  project,  implying 
that  at  the  end  the  goals  were  clear,  but  that  it  took  awhile  for  this  to  happen.  It  is  possible  that  they 
consider  the  lack  of  objectives  an  issue  of  management  style. 

Brunner,  C.  and  Henriquez,  A.  (1991)  Evaluation  of  Stevens  Institute  of  Technology's  "Integration 
of  Computers  into  Mathematics  Education"  Project.  New  York,  NY:  Bank  Street  College  of 
Education,  Center  for  Children  and  Technology 

11.  Appendix  III:  Technological  Innovation  in  a  High  School  Mathematics  Department:  A  Cultural 

and  Structural  Analysis 

Excerpts  from  the  Ph.D.  Dissertation  of  Keith  F.  Allum,  Department  of  Sociology,  Princeton 
University. 

Discussing  the  interaction  between  the  CIESE  project  and  "Conant  High  School",  one  of  the  five 
initial  sites  collaborating  with  Stevens: 

"Simply  stated,  CIESE  philosophy  matched  Conant  culture.  The  preliminary  comments  made  by  the 
project  director  and  the  subsequent  discussions  they  sparked,  set  a  tone  which  resonated  within  the 
bounds  circumscribed  by  the  cultural  forms  of  the  mathematics  department.  The  confession  of 
fallibility  (by  the  CIESg  project  director)  emboldened  the  teachers  to  contribute  to  the  planning  and 
direction  of  the  project  .  Moreover,  the  project  staff's  treatment  of  teachers  as  professionals 
encouraged  and  strengthened  teacher  commitment  to  the  project.  The  teachers  thus  gained  a  sense  of 
joint  ownership  which  led  them  to  increase  their  commitment  to  the  project.  Most  prior  inquires 
into  externally  directed  change  reveal  that  there  is  seldom  a  period  of  consultation  with  the 
classroom  teachers  and  perhaps  for  this  reason,  such  strategies  often  failed. 

Under  other  circumstances,  such  an  admission  by  a  project  director  would  indicate  poor  planning 
and  eminent  disaster.  Yet  it  may  be  argued,  and  some  data  bear  this  out,  that  the  external  expert's 
denial  of  omniscient  expertise  paradoxically  lead  to  a  greater  likelihood  of  success." 

"It  is  important  to  note  that  some  members  of  the  mathematics  staff  commented  that  CIESE  merely 
allowed  them  to  do  what  they  had  always  intended.  This  claim  allowed  teachers  to  define 
themselves  as  innovators  and  the  CIESE  staff  as  fellov/  change  agents." 

"Teachers  could  not  integrate  technology  into  their  classroom  practice  without  (1)  institutional 
support  form  the  district  (i.e.,  time  and  resources)  and  (2)  knowledge  and  experience  in  the  form  of 
training.  Both  were  obtained  through  participation  in  the  CIESE  project.  CIESE  virtually 
guaranteed  support  as  a  precondition  for  participation." 

Quote  by  Allum  of  a  teacher  referred  to  as  "Ted":  "Stevens  did  a  very  smart  thing  by  getting  the 
Board  to  commit  themselves  to  the  project.  Commitment  from  the  Board  is  really  crucial  in  a  major 
i^xOgram  like  this." 
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"Generally  speaking,  the  CIESE  project  has  served  to  focus  attention  on  the  importance  of 
mathematics  and  thus  increased  the  status  of  the  discipline  in  the  school  and  community  and 
expanded  options  of  teachers  in  presenting  the  subject  matter  to  their  students." 

"The  school-wide  survey  indicated  that  91%  of  the  student  body  perceived  computer-based 
technology  to  be  a  positive  addition  to  the  school  environment  and  84%  indicated  a  desire  to  use  the 
PC*s  more  often  (especially  in  Mathematics  and  English  classes).  The  receptivity  of  students  led 
teachers  to  use  computer  applications  as  a  technique  to  capture  interest  and  gain  cooperation," 

"Classroom  management  cannot  be  sacrificed  in  the  name  of  innovation.  By  contributing  to  the 
control  and  motivation  of  students,  microcomputers  have  been  perceived  as  useful  tools  in  the 
management  of  the  classroom. " 

"The  CIESE  Project  continues  to  develop  'successful  models*  of  computer  implementation  in  the 
hope  that  other  districts  will  be  able  to  emulate  a  tested  and  effective  prototype.  Plans  for  assess- 
ment and  evaluation  were  built  into  the  original  project  design.  Now  in  its  third  year,  qualitative 
and  quantitative  studies  are  being  conducted  to  discover  the  effects  of  classroom  computer  use  on 
student  achievement  and  school  organization." 

Allum,  Keith  (1991)  Technological  Innovation  in  a  High  School  Mathematics  Department:  A 
Structural  and  Cultural  Analysis,  Dissertation  in  partial  fulfillment  of  the  Doctor  of  Philosophy 
requirements,  Department  of  Sociology,  Princeton  University 


12.  Appendix  IV:  Software  Used  in  the  Sections  Taught  with  the  Use  of  Computers 

Discovery  Learning  in  Trigonometry,  developed  by  John  B.  Kelley  and  distributed  by  CONDUIT. 

Geometry  Proof  Tutor,  distributed  by  Advanced  Computer  Tutoring,  Inc. 

Geometric  Supposer:  Triangles,  developed  by  Education  Development  Center,  Inc.  and  distributed 
by  Sunburst  Communications. 

Green  Globs  and  Graphing  Equations,  developed  by  Sharon  Dugdale  and  George  Kibbey  and 
distributed  by  Sunburst  Communications. 

Mathematics  Exploration  Toolkit,  developed  by  WICAT  Systems,  Inc.  and  distributed  by  IBM 
Corporation. 

Ratios  and  Proportions^  developed  by  Intellectual  Software  and  distributed  by  Queue. 


52 


THE  EFFECTS  OF  QUES'nONING  ACCESS  IN  COMPUTER-BASED  INSTRUCTION 

Joan  C.  Lee  and  Richard  Dennis 
University  of  Illinois  at  Urbana-Champaign 

ABSTRACT 

The  pn  sent  stiKiy  examines  the  effects  of  questioning  access  on  retention  of  information  in  a  CBI 
lesson.  The  subjeci>  were  15  high  school  students  assigned  to  3  experimental  situations:  (1)  open 
accessible  questions  (no  specific  sequence),  (2)  prescribed  questions,  and  (3)  prescribed  topic  presentation. 
No  differences  in  achievement  were  attained  as  a  function  of  the  three  instructional  methods,  Tliis  result 
may  be  attributed  to  the  small  sample  size.  Findings  show,  however,  that  one  student  who  selected  a 
question  of  interest  eveiy  single  time  gready  improved  her  achievement,  though  other  students  seldom 
made  a  jump  (i,e.,  non-linear  sequence  of  selection)  during  instruction.  If  a  jump  occurred,  the  adjacent 
questions  were  likely  to  be  probed  next;  otherwise,  the  prior  sequence  was  resumed.  It  was  also  found  that 
providing  questions  in  this  CBI  lesson  did  not  improve  students'  self-questioning  skills  on  a  new  subject 
matter, 

INTRODUCTION 

Reading  expository  text  has  been  one  major  method  of  acquiring  knowledge  in  the  conventional 
academic  setting.  As  a  technology-based  instructional  mode,  computer-based  instruction  (CBI)  lessons  also 
present  students  textual  information  for  learning.  The  main  focus  of  this  study  is  to  design  a  CBI  lesson  to 
present  information  in  the  form  of  questions  and  answers  in  order  to  facilitate  the  siudcra's  processing  of 
expository  text. 

Hall  (1983)  and  Merrill  (1987)  believe  that  the  type  of  questions  presented  to  a  learner  is  a  crucial 
variable  in  designing  courseware.  This  particular  study  is  therefore  created  with  two  objectives.  One 
objective  is  that  we  intended  to  improve  the  form  of  questions  in  order  to  induce  a  learner's  selective 
attention  during  the  encoding  process  of  information.  Secondly,  we  attempted  to  apply  the  learner-control 
strategy  to  the  sequencing  of  questions  so  that  a  learner  can  be  actively  engaged  in  his  or  her  learning 
process.  The  theorization  of  selective  attention  is  derived  from  the  information  processing  model. 
Researchers  in  the  line  of  reading  comprehension  have  discussed  that  prequestions  can  be  used  to  induce 
readers'  selective  attention.  With  respect  to  sequencing  of  questions,  our  prequestions  are  characterized  by 
their  open  accessibility  (i.e,,  non-linear  sequence),  Tliis  approach  is  derived  from  the  research  in  locus  of 
control.  Additionally,  a  top-level  structural  map  was  added  to  aid  students  to  learn  through  the  lesson.  The 
use  of  the  map  is  adapted  from  an  adjunct  aid  developed  by  Meyer  (1975).  The  related  research  will  be 
reviewed  in  the  next  section. 
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REVIEW  OF  LITERATURE 


Selective  Attention 

Most  of  the  instructional  research  today  is  based  on  an  information  processing  niodei  of  human 
memory  developed  by  Shiffrin  and  Atkinson  in  1969.  The  model  assumes  that  the  human  memory  system 
consists  of  three  memory  buffers:  sensory  registers,  shon-term  memory,  and  long-term  memory.  It  is 
assumed  that  information  travels  from  sensory  registers  to  shon-term  memory  where  it  is  encoded,  Ii  is 
then  transferred  from  short-term  memory  to  long-term  memory  -  the  permanent  storage  of  the  information. 
Learning  is  a  process  by  which  information  is  encoded  and  transferred  from  shon-term  memory  to  long-term 
memory  (Hannafin  &  Rieber,  1989),  In  fact,  shon-term  memory  has  a  limited  processing  capacity  , 
because  it  can  only  hold  up  to  7  plus  or  minus  2  units  of  information  (Miller,  1956),  Therefore,  the  learner 
must  selectively  scan  the  incoming  information  in  sensory  registers  and  move  important  information  from 
sensory  registers  to  shon-term  memory  (Kumar,  1971),  Anderson  (1970)  has  referred  to  this  as  "selective 
attention".  In  addition  to  individual  factors,  some  external  factors,  such  as  how  information  is  presented  and 
organized  may  affect  one's  selective  encoding  process  (Hannafin  &  Rieber,  1989),  Thus,  Gagne  &  Briggs 
(1988)  point  out  that  one  of  the  most  imponani  instructional  objectives  is  to  focus  the  learner's  attention 
on  relevant  information  in  the  instructional  presentation*  That  is,  selective  attention  can  strengthen  one's 
encoding  process  of  information.  Poorly  coded  information  may  not  get  to  be  transferred  to  long-term 
memory  and  is  therefore  forgotten. 

Effect  ofPrequestions 

Wager  &  Wager  (1985)  state,  "Questions  serve  three  general  functions  in  learning:  (a)  To  establish 
and  maintain  attention,  (b)  to  facilitate  encoding,  and  (c)  to  provide  for  rehearsal"  (p,  4),  In  the  line  of 
reading  comprehension  research,  adjunct  questions  are  defined  as  questions  interspersed  throughout  textual 
materials  that  students  are  instructed  to  answer  (Hamaker,  1986).  The  adjunct  questions  are  helpful  to 
maintain  students*  attention  and  focus  on  important  information  for  reading  longer  segments  of  text  (Bean, 
1985).  One  of  the  major  findings  on  adjunct  questions  research  summarized  by  Hamaker  (1986)  is  that 
adjunct  questions  produce  a  fiacilitating  effect  on  memory  performance,  Hamiltom  (1985)  wlw  reviewed  a 
number  of  studies  also  finds  that,  regardless  of  the  placement  before  or  after  the  text,  questions  aid  retention 
when  tested  in  immediate  recall,  delayed  recall,  and  transfer, 

A  "prequestion"  is  one  type  of  adjunct  question  that  precedes  the  text  to  be  read,  Anderson  and 
Biddle  (1975)  report  that  prequestions  have  a  positive  effect  on  retention  of  relevant  information.  But  at  the 
same  time,  they  suggest  that  such  prequestions  may  actually  retard  retention  of  unrelated  information. 
Similarly,  Kumar  (1971)  repon  that  students  lend  to  achieve  belter  retention  of  relevant  information  cued  by 
prequestions  and  do  poorly  on  recalling  irrelevant  information.  These  results  confirm  the  role  of  selective 
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attention  in  the  information  processing  model.  Because  of  selective  attention  induced  by  prequestions. 
irrelevant  information  is  filtered  before  it  enters  short-term  memory,  Kumar  (1971)  claims  Uiat 
prequestions  maximize  the  use  of  short-term  memory.  Ke  states,  "The  content  of  the  short-term  memory 
was  the  question-relevant  material  and  the  depth  of  processing  was  achieved"  (Kumar,  1971,  p.  408),  In 
fact,  discarding  irrelevant  information  during  the  encoding  process  is  fairly  economic  with  regard  to  the 
limited  capacity  of  short-term  memory  (Kumar,1971).  In  Reynolds  and  Anderson's  (1982)  study,  it  is  found 
that  as  a  result  of  readers'  selective  attention,  readers  pay  more  attention  to  and  leam  more  infonnation 
relevant  to  prequestions.  West,  Fanner,  &  Wolff  (1991)  summarize  Hamilton's  review  (1987)  and  report 
thai  even  the  effects  of  high-level  prequestions  such  as  application  or  synthesis  prequestions  are  superior  to 
llie  effect  of  high-level  postquestions,  Howevw,  research  has  also  shown  that  during  the  encoding  process 
of  information,  one's  frequent  and  overt  responses  to  questions  may  interrupt  his  or  her  semantic  processing 
of  information  (Wager,  1982).  Questions  requiring  no  response  may  become  metacognitive  in  nature, 
because  students  are  able  to  process  learning  materials  in  their  own  ways  rather  than  be  judged  by 
correctness  or  incorrectness  of  their  answers  (Hannafui,  1989). 

Lociis.Qf  Control 

In  recent  years,  cognitive  psychologists  put  much  emphasis  on  the  notion  that  a  learner  is  not  a 
spectator,  but  a  purposeful  participant  in  the  learning  process.  There  has  been  extensive  instructional 
research  which  attempts  to  adapt  individual  differences  to  the  instructional  design.  Researchers  focusing  on 
locus  of  control  have  been  investigating  the  issue  of  whether  program-controlled  instruction  is  superior  to 
learner-controlled  instruction  or  vice  versa.  Under  the  program-contfolled  condition,  sequence  options 
include  linear  and  conditional  branching  which  is  based  on  the  learning  speed  and  on-task  performance.  The 
Icamcr-conu-oUed  method  is  non-linear  in  nature.  One  approach  is  to  permit  learners  to  access  to-bc-leamcd 
•  information  through  numerous  options  in  terms  of  menus  and  indexes  (Hannafin,  1989).  The  other 
approach  is  referred  to  by  Steinberg  (1989)  as  "dynamic  locus  of  control,"  whereby  useful  problem-solving 
tools  are  accessible  throughout  the  lesson  so  that  students  are  able  to  use  their  own  method  to  learn  the 
lesson  content, 

Merrill,  Sclineider,  &  Fletcher  (1980)  suggest  that  individual  differences  can  be  accommodated  by 
allowing  learners  to  contfol  the  sequence  of  instruction.  However,  research  has  yielded  mixed  results 
concerning  the  influence  of  learner  contfol  on  the  quality  of  learning.  Lee's  (1990)  study  shows  that  a 
leamer-conu-oUed  method  increases  students'  metacognitive  and  cognitive  abilities  better  than  a  program- 
conu-oUed  method.  Tennyson  (1980)  indicates  that  students  in  a  learner-controlled  strategy  master  the 
objective  of  the  lesson  in  less  time  than  those  in  a  program-conu-oUed  strategy.  On  the  other  hand,  Clark 
(1984)  finds  that  students  tend  to  choose  the  type  of  instruction  which  is  not  helpful  to  their  learning 
process.  Learners  may  not  make  a  good  judgement  on  what  they  need  from  the  instruction  in  order  lo 
improve  their  performance  (Tennyson,  Park,  &  Christensen,  1977).  Therefore,  Steinberg  (1989)  claims 
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that  a  medialed-leamer  support  instruction  optimizes  the  balance  between  a  program-controlled  and  a  leamer- 
controlled  instruction.  The  mediaied-leariner  support  instruction  takes  the  intermediate  position  in  a 
spectrum  that  has  a  program-controlled  instruction  at  the  one  end  and  a  leamer-ccMitroUed  instruction  at  the 
otlier  end  of  the  spectrum.  Boyd,  Douglad,  &  Lebcl  (1984)  designed  a  mediated-leamer  support  system 
which  included  some  memory  aids  such  as  a  review,  hints,  glossaries,  and  charts  in  a  learner-controlled 
lesson  and  found  that  the  groups  learning  with  memory  aids  outperformed  non-aid  groups.  Similarly, 
Steinbexg,  Baskin,  &  Matthews  (1985)  concluded  that  although  memory  charts  were  not  equally  effective 
for  all  individuals  at  alt  stages  of  learning,  overall  these  organizational  memory  charts  aided  students 
problem  solving  performance. 

An  Organizaiiongl  Aid;  Tpp-Uvgl  Structurs 

Meyer  (1975)  has  developed  a  top-level  structure  that  underlies  the  main  ideas  and  details  of  the 
text  Main  ideas  occur  at  the  top  level  of  the  hierarchical  structure.  They  are  better  retained  and  also  allow 
for  the  retention  of  related  details.  Wilhiie  (1983)  states,  "Superordinate  units  are  critically  important  to  the 
memory  of  the  passage  as  a  whole  and  that  through  these  units,  access  to  subordinate  units  is  achieved"  (p. 
235).  In  Meyer,  Brandt,  &  Bluth's  study  (1980),  there  is  increasing  probability  that  students  remember 
more  informatiwi  if  readers  learn  the  strategic  use  of  the  top-level  structure  as  they  read  the  text,  Mayer 
(1984)  has  suggested  thai  one  type  of  aid  for  information  processing  should  serve  as  a  cue  to  learners  as  to 
how  to  organize  information.  Brown  &  Smiley  (1977)  points  out  that  the  purpose  of  using  any  learning 
aids  is  to  compensate  for  readers'  metacognitive  skills.  In  other  words,  the  top-level  smjcture  as  an 
organizational  aid  does  not  only  facilitate  readers  to  remember  and  comprehend  the  text,  but  also  enhances 
readers*  metacognitive  skills. 

Tripp  and  Roby  (1990)  suggest  Uiat  advanced  organizers  and  metaphors  can  provide  the  stnicturc  of 
computerized  learning  materials.  Anderson  &  Armbruster  (1985)  argue  thai  \h&  sliuciure  of  text  should  be 
made  explicit  to  Uie  readers  for  instructional  purposes.  Schloss,  Schloss,  &  Cartwright  (1984)  find  that 
questions  added  to  a  text  as  signals  are  significantly  superior  to  a  highlighting  format  and  also  superior  to  a 
straight  text  when  tested.  To  combine  prequestions  with  a  top-level  structure  strategy,  Wilhiie  (1983) 
employs  quiz  prequestions  to  make  the  top-level  structure  of  die  passage  clear  to  students. 

R^garcti  Qugslions 

In  a  traditional  CBI  lesson,  students  encounter  ceilain  prescribed  questions  after  tiiey  read  some 
information  and  usually  are  not  allowed  to  advance  through  the  lesson  unless  their  answers  are  correct 
However,  the  CBI  lesson  in  die  present  study  is  created  differently  in  terms  of  format  and  sequence  of 
questions.  The  format  of  questions  is  such  that  each  question  precedes  a  segment  of  text  containing  the 
answer  to  the  question.  Tliai  is,  die  answer  has  been  provided  by  die  lesson.  This  type  of  questions  is 
designed  in  light  of  Uic  attentional  properly  of  prequestions.  Based  on  the  learner-control  strategy,  there  is 
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no  specific  sequence  of  questions  in  this  CBI  lesson.  All  questions  (i.e.,  prequestions)  are  accessible  by  a 
menu  rather  than  by  the  lesson  designer  deciding  when  and  where  and  how  many  to  be  encountered. 
Students  may  choose  any  prequestion  of  interest  in  the  menu  and  read  the  answer  on  the  next  screen. 
Moreover,  as  an  organizational  aid.  the  top-level  structure  map  is  created  to  aid  students  in  understanding 
main  topics  presented  in  the  lesson. 

The  treatments  in  terms  of  three  types  of  instructional  methods  are  tested  in  the  present  study. 
One  treaunent  is  that  the  lesson  content  is  presented  by  open-accessible  (non-linear)  prequestions.  The 
other  treatment  uses  prescribed  (linear)  prequestions  to  present  the  same  lesson  content  Instead  of  uses  for 
prequestions,  i»^scribed  (linear)  topics  are  used  in  another  treatment  Therefore,  three  research  questions  in 
this  study  ani  formed  as  follows:  (1)  Will  diree  types  of  instructional  methods,  non-linear  prequestions. 
linear  preqi!<JStions,  and  linear  topics  produce  significant  differences  in  students'  learning  outcomes?  (2) 
Which  foiiTTiat,  non-linear  or  linear  prequestions  will  better  improve  students'  performance?  (3)  Can 
providing  prequestions  train  students  to  develop  self-questioning  skills?  FurthenDC^e,  a  qualitative  analysis 
on  students'  cognitive  styles  of  learning  is  conducted. 

METHOD 

The  subjects  in  this  experiment  were  15  senior  high  school  students  who  enrolled  in  the  summer 
internship  program  in  1992  at  US  Army  Construction  Engineering  Research  Laboratory  in  Champaign, 
Illinois.  The  students  were  randomly  assigned  to  one  of  the  three  experimental  conditions.  Thus,  there 
were  five  subjects  in  each  experimental  condition. 

Materials 

The  text  used  in  this  experiment  was  a  scientific  text  about  earthquakes.  The  scientific  text, 
expository  in  nauire,  aims  at  more  information  on  earthquakes  for  Midwest  students,  because  earthquakes 
may  occur  in  the  Midwest.  The  content  was  adapted  from  several  earth  science  textbooks  for  high  school 
students.  This  CBI  lesson  was  programmed  by  using  HyperCard  with  three  different  versions  designed  for 
three  experimental  groups.  Each  group  received  one  version  of  the  lesson.  The  fu-st  version  contained  open 
accessible  (non-linear)  questions  in  a  menu  (see  Figure  1). 
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Click  on  any  question  of  interest  end  reed  the  aMver, 


1  Can  earthquakes  chancie  natural  l;mdscap*s? 

2  Can  earthquakes  occur  in  the  sea? 

3  Can  earthquakes  occur  in  the  Mi<Jwest? 

4  Can  plate  tectonics  acco»int  for     earthquake  oc^jurrences? 

5  Hov  are  earthquake  vaves  recorded? 

6  Hov  do  people  interpret  ratings  on  the  Richter  Scale? 

7  Hov  do  people  measure  earth<[^iakes? 

8  How  vas  the  Midwest  geology  formed? 

9  How  vas  the  San  Andrea  Fault  formed? 

1 0  In  what  ways  do  crustal  plates  of  the  Earth  move  ? 

1 1  What  are  "faults"  ? 
t2  Vhat  are  L  waves? 

13  Vhat  are  P  w^ves? 

14  Vhat  are  $  waves? 

15  Vhat  are  sei<>mic  waves? 

6  Vhat  are  twelve  levels  "-^  t*>rms  of  Mercalli  Intensitu  Scale? 


EicirfiJL-  Menu  of  Prcqucstions  in  Uie  CBI  Lesson. 

There  were  30  questions  sl\  slanting  with  words  such  as  can,  liow,  what,  why,  and  where.  An  answer  was 
presented  on  the  screen  right  after  a  question  had  been  chosen  in  the  menu.  The  length  of  an  answer  was 
approximately  70  words.  On  the  screen  prcscniing  ihc  answer,  the  question  in  bold  was  displayed  again.  A 
graphic  illustration  was  shown  on  tJie  screen  as  well  (se«  Figure  2).  The  second  version  of  the  lesson 
contained  the  same  questions  as  ths  first  version.  But,  the  sequence  of  questions  was  predetermined  by  the 
lesson  as  to  where  the  learner  was  allowed  to  access  questions  in  a  linear  sequence.  The  third  version  of  the 
lesson  contained  a  menu  of  topics  in  the  same  sequence  as  the  second  version.  Three  versions  presented  the 
same  ii^ormaiion  (i.e..  io-tx^-read  text)  which  is  embedded  in  the  answers  in  the  first  and  the  second 
versions;  and  in  explanations  about  the  topics  in  the  third  version. 

A  top-level  structure  map,  an  organizational  aid,  was  only  added  to  the  first  version  of  the  lesson 
only  to  display  the  main  eartliquake  topics  in  the  lesson.  AH  questions  in  the  menu  were  organized 
hierarchically  into  a  map  (see  Figure  3).  Five  major  questions  eliciting  the  most  important  information 
were  regarded  as  main  ideas  and  were  displayed  at  the  top  level  of  the  hierarchical  map.  They  were:  (1) 
**What  causes  eartliquakcs?"  (2)  '*What  are  the  effects  of  earthquakes?"  (3)  "How  are  earthquakes  detected?"  (4) 
"How  are  earthquakes  measured?"  and  (5)  "V/lwre  are  earthquake  zones?".  Some  questions  which  elicited  the 
second-level  informatioiJ  (i.Ce,  they  were  not  main  idea  questions  or  detailed  questions)  were  positioned 
below  the  main  idea  in  tlie  hiearchy  (e.g.,  the  question  25).  Tnc  detailed  questioas  were  then  placed  at  the 
bottom  level  of  Uic  hicarchy  (e.g..  the  questions  9,3, 6, 15,  and  20).  The  map  initially  contained  only 
structural  lines  witii  question  marks  which  represented  questioji  numbers.  After  the  student  had  read  one 
question,  one  question  mark  was  replaced  by  tlic  question  number  associated  with  that  question.  Students 
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could  see  one  particular  position  blinking  on  the  map  screen,  while  this  repiacenricnt  was  taking  place.  The 
blinking  effect  was  used  to  draw  subjects'  attention  to  the  lop-level  suiicture  map.  At  the  end  of  the  lesson, 
the  map  was  filled  by  question  numbers  at  the  proper  places.  Stud«its  read  questions  or  answers  on  one 
screen  and  viewed  the  map  on  the  other  screen  at  the  same  lime. 


^  Can  earthquakes  chfinqe  natural 
landscapes? 

Strong  shocks  from  an 
earthquake  loosen  fracture 
and  veather  rocks  high  on 
the  mountain,  causing  them  to 
fall.  As  the  falling  rocks  gather 
momentum,  they  will  loosen 
more  debris  and  thunder  down 
the  slope  into  a  river.  Lakes, 
ponds,  and  other  standing  bodies  of 
water  may  also  be  affected  by 
earthquake  vi  brations. 


The  natural  dam. 
Earthquake  Lake,  created 
by  landslide  Madison 
Canyon  and  Madison  River. 


MENU 


Figure  2.  To-Be-Leamed  Text  and  Graphics. 


How  are  earthquakes  measured?  J- Scale  ?  ?@? 


Plate  Theory  ??(§) 
IntraPlate  Earthquake3(3^  7 


7  7 


How  are  earthquakes  detected?  | — 


P  Form  of  Energy (f^?  7  7  7 


What  are  the  effects  of  earthquakes? 


^  Instrument  7  7 

Land  ?@? 
Sea    7  7 

I  WherearetheearthquakezonesTl|"|r  Island  7 


p  m 


Continent 


9  *? 


Figure  3.  Top-Level  Structure  Map. 
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Thirteen  factual  lest  questions  covering  the  instructional  content  in  the  pretest  were  repeated  in  the 
posttest  but  in  a  different  format  so  as  to  eliminate  the  practice  effect.  Pretest  questicMis  were  in  the  Due- 
false  format,  whereas  the  posttest  questions  were  multiple-choice  questions.  As  a  transfer  lest,  subjects 
were  requiied  to  generate  questions  regarding  earthquakes  in  the  pretest  and  tornadoes  in  the  posttesu 

DESIGN 

The  experiment  used  a  one-way  ANCOV  A  between -groups  design.  In  this  experiment,  the 
independent  variable  was  the  insiructional  method,  the  dependent  variable  was  posttest  scores  and  the 
covariate  was  the  pretest  scores.  In  ihjs  case,  the  covariate  (i.e.,  pretest  scwes)  was  to  adjust  the  posttest 
means  for  any  initial  differeiKcs  that  may  have  been  present  in  the  pretest  in  order  to  yield  a  more  powerful 
test. 

In  the  study,  there  were  three  experimental  conditions:  open-accessible  (non-linear)  prequestion 
group  (OP  group),  linear  prequestion  group  (LP  group),  and  linear  topic  groups  (LT  group).  As  mentioned 
in  the  preceding  section,  two  versions  of  the  CBI  lesson  contained  the  same  prequeslions  that  differed  in 
sequence.  The  one  with  open  accessibility  of  prequeslions  was  given  to  the  OP  group.  Ttie  LP  group 
received  the  other  version  with  prescribed  prequcstions.  The  third  group,  the  LT  group  which  served  as  the 
control  group,  received  the  third  version  in  which  topics  were  accessed  in  a  [described  sequence.  Only  the 
OP  group  was  jxovided  witii  a  top-level  structure  map  (see  Table  1)  in  the  lesson. 

Table  1 

IXscripiions  of  EMpgrimfintel  Conditions 


Open- Accessible  Prequestion  Linear  Prequestion  Group  Linear  Topic  Group 
Group 

(OP  Group)                           ClP  Group)  (LT  Group) 

Prequeslions                          Prcquestions  Topical  words 

Non-Linear  Sequence                    Linear  Sequence  Linear  Sequence 
A  Structure  Map 


Each  subject  was  required  to  lake  the  pretest  and  the  posttest.  Subjects  were  tested  either 
individually  or  in  pairs.  There  was  no  time  limil  in  ilic  CBI  lesson.  The  individual  learning  time  dial  each 
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subject  spent  in  selecting  questions  and  reading  answers  was  recorded.  The  computer  kept  track  of  the 
sequences  in  accessing  quesiicMis  created  by  the  OP  group.  There  were  three  steps  used  by  these  three 
groups  to  complete  the  lesson: 

(1)  The  subject  selected  a  question  /  a  topic  in  the  menu  (in  the  case  of  the  LP  group  and 
LT  group,  subjects  sequentially  selected  the  next  question  /  topic  in  the  menu).  Then,  the 
computer  displayed  the  lo-be-leamcd  text  on  the  screen. 

(2)  After  having  read  the  iext»  the  subject  returned  to  the  menu. 

(3)  Repeat  (1)  until  all  questions  had  been  read. 

Correct  answers  to  the  factual  test  items  in  the  pretest  and  postlest  questions  were  scored  as  one 
point,  and  zero  points  for  wrong  answers.  Student-generated  questions  in  the  pretest  and  tlie  posttest  were 
compared  to  the  questions  presented  in  the  CBI  lesson.  Two  raters  examined  student-generated  questions  to 
determine  whether  or  not  the  questions  were  semantically  simiiar  to  the  CBI  questions.  Ii  was  assumed  that 
students  who  learned  about  earthquakes  through  questions  were  likely  to  generate  questions  simiiar  to 
earthquake  questions  but  in  the  context  of  tornadoes.  Therefore,  a  student-generated  question  similar  to  a 
CBI  question  was  said  to  be  a  qualified  question.  The  number  of  qualified  student-generated  questions  was 
calculated. 

RESULTS  AND  DISCUSSION 

Means 

Table  2  presents  the  descriptive  statistics  of  pretest  and  posttest  scores.  A  t-lcst  was  conducted  to 
determine  whether  there  is  a  significant  difference  between  the  pretest  scores  and  the  posttest  scc^'cs.  The 
result  shows  that  this  CBI  lesson  produces  a  significant  difference  in  subjects'  performance  on  pretest  and 
posttest  (((14)  =  4.60,  ***j^<  .001). 
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Table! 

Descriptive  Statisucs  on  Pretest  and  Posttest  Scores 


Pretest  posttest 


Sample 

N 

M 

SD 

M 

SD 

15 

6.53 

2.26 

9.20 

2.51 

Experimental  Groups: 
OP  Group 

5 

6.40 

1.52 

9.20 

1.48 

LP  Group 

5 

6.80 

3.34 

9.20 

3.96 

LT  Group 

5 

6.40 

2.07 

9.20 

2.04 

Note.  Maximum  score  =  13. 


It  was  also  found  that  pretest  scores  were  correlated  with  posttest  scores  (l=  .56).  Therefore,  31% 
(I.e.,  the  square  of  the  correlation  coefficient)  of  variance  on  posttest  scores  was  atuibuted  to  variability  on 
pretest  scores.  ANCOV A  was  conducted  to  remove  this  proportion  of  the  variance  from  the  error  term. 
However,  one  of  assumptions  underlying  ANCOV  A  for  one  covariate  requires  equal  population  regression 
slopes  for  all  groups.  In  this  study,  the  nature  of  the  linear  relationship  is  not  the  same  for  the  three 
experimental  groups.  Figiu-e  3  reveals  that  the  slopes  are  not  equal.  Tlien,  ANCOVA  is  not  appropriate. 
Alternatively,  a  one-way  ANO VA  on  the  gain  scores    the  differences  between  posttest  scores  and  pretest 
scores,  was  conducted  to  test  whether  the  instructional  metliod  causes  differences  in  subjects'  performance. 
No  significant  difference  was  obtained  (MSb  =  .30,  MSc=  5.90,  E{2.12)  =  .05), 


Since  this  was  a  pilot  study,  a  small  sample  was  used.  Because  of  the  small  sample  size,  a  power 
problem  occurred.  In  deciding  the  number  of  subjects  per  group  needed  to  have  adequate  power,  it  was 
assumed  that  a  medium  effect  size  of  .25  and  a  70%  chance  (power  =  ,70)  of  finding  a  difference  should  be 
achieved.  Therefore,  42  subjects  per  group  were  needed  to  obtain  .05  significance  level  and  32  subjects  per 
group  to  obtain  ,10  significance  level  (Stevens,  1990). 

Pgrfprmang^^s  on  Fag^uai  0»g$uoPS  of  Prgtgsi  and  Posticgi 

As  mentioned  in  the  preceding  section,  the  pretest  contained  the  same  factual  questions  as  the 
posttcst.  Figure  4  shows  that  the  posttest  scores  scatter  around  higher  scores  such  as  10, 1 1, 12,  and  13 
points  than  the  pretest  scores  do. 


15 
14 
13 
12 

a  10 
9 
8 
7 
6 
5 
4 
3 
2 
1 
0 


i 


□  Pretest 
■  Posttcst 


Jllk 


m  m 


4  5 


6  7  8 
Test  Scores 


10     11     12  13 


Figure  4.  Frequency  Distribution  of  Two  Test  Scores, 


The  number  of  correct  answers  to  lest  questions  across  the  two  tests  was  compared  (see  Figtire  5  ). 
None  of  the  subjects  answered  test  question  2  correctly  in  the  pretest  But,  this  situation  was  greatly 
improved  in  the  posttest.  Similarly,  the  number  of  correct  answers  to  test  questions  1,  3, 4, 6,  and  9 
increased  in  the  posttest.  By  contrast,  the  performance  on  test  questions  7, 10, 11, 12,  and  13  became 
poorer  in  the  posttest  than  in  the  pretest  Although  the  two  tests  of  different  format  were  intended  to 
eliminate  tlie  practice  effect,  it  was  suspected  that  the  decreased  number  of  correct  answers  in  the  posttest  on 
the  same  test  questions  were  due  to  the  lower  chance  of  answering  multiple-choice  questions  correctly  than 
answering  true-or-false  questions  in  the  pretest  correctly. 
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Test  Questions 

Figure  5.  Total  Number  of  Correct  Answers  on  Two  Tests 

Bg  Uaming  Timg  and  Rgading  TimQ 

The  learning  lime  includes  the  lime  spcni  in  selecting  questions  in  the  menu  and  the  lime  spent  in 
reading  the  texts  contained  in  answers.  In  other  words,  the  learning  time  was  time  spent  in  learning 
through  ihe  lesson.  Table  3  depicts  the  average  learning  time  over  three  groups. 

Tables 

Descriptive  statistics  of  Average  Learning  Time  bv  Groups 


Average  Learning  Time  in  Minutes 


Experimental  Groups  Min.  Max.  M  SD 


OP  Group 

26.30 

35.73 

29.12 

1.70 

LP  Group 

18.00 

29.02 

25.18 

4.38 

LT  Group 

15.97 

32.38 

23.59 

7.20 

The  OP  group  had  a  longer  lesson  learning  time  than  the  other  two  groups.  This  was  because  that 
OP  subjects  selected  questions  to  read  while  tlie  LP  and  LT  subjects  read  questions  in  order.  Moreover,  the 
data  indicate  that  there  is  a  greater  variability  in  learning  time  for  the  LT  group  (SD  =  7.20)  than  for  the 

other  two  groups.  It  was  also  found  that  the  OP  subjects'  learning  time  was  highly  correlated  with  their 
posttest  scOTes  (c  =  .79,  see  Table  4).  The  longer  the  OP  subjects  spent  in  learning  with  the  CBI  lesson, 
the  better  their  performance. 
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Table  4 

Correlation  Between  Posttcst  and  Leamini?  Time  by  Groups 


Postiest  Scores 

OP  Group 

LP  Group 

LT  Group 

Learning  Time 

.79 

-.15 

-30 

Since  a  card  contained  a  segment  of  to-be-read  '  x^*.,  the  reading  time  was  recorded  on  a  single  card 
basis.  The  average  riding  time  spent  by  the  three  groups  on  reading  text  through  the  lesson  is  shown  in 
Figure  6.  A  greater  amount  of  time  was  spent  in  reading  the  texts  on  cards  16, 23,  and  29  than  the  other 
cards.  As  a  matter  of  fact,  card  16  and  card  23  contained  longer  segments  of  text  Card  16  contained 
information  about  the  twelve  levels  of  Mercalli  Intensity  Scale  and  card  23  described  what  happened  in  San 
Francisco  in  1989.  The  text  on  card  29  was  short  but  described  the  location  of  the  New  Madrid  Fault. 
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Card  Numbers  in  Lesson 


□   OP  Group       A   LP  Group        ©   LT  Group 


Figure  6.  Average  Reading  Time  by  Groups  over  Cards. 


The  texts  of  these  cards  were  more  descriptive  than  expository  in  nature.  They  involved 
descriptions  about  people,  events,  damages,  and  places.  Ii  was  found  that  subjects  paid  attention  to  reading 
the  text  even  though  the  text  was  lengthy.  Additionally,  Figure  6  reveals  that  the  LP  group  spent  less  time 
in  reading  the  text  than  the  other  two  groups.  In  reference  to  the  means  of  the  pretests  for  the  three  groups 
(see  Table  2  in  the  previous  page),  the  LP  group  had  the  highest  mean  score  among  the  three.  It  confirms 
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lo  the  schema  research  that  people  with  well-developed  prior  knowledge  of  the  lesson  content  process  a  text 
more  efficiently  and  rapidly. 


Selection  Time  and  Sequencing 

For  the  OP  subjects,  their  selection  time  accounted     a  portion  of  their  learning  lime,  because 
they  selected  questions  from  a  niertu  to  read.  Figure  7  shows  that  the  OP  subjects  spent  relatively  less  time 
in  deciding  which  question  to  read  next,  ever  since  after  ihey  decided  which  question  to  read  first  In 
addition,  ii  reveals  tliat  subject  3  spent  a  greater  amount  of  time  in  selecting  questions  to  read  than  the  other 
OP  subjects.  Subject  3's  selection  time  dramatically  varied  from  one  question  to  another.  In  reviewing 
relevant  data,  it  was  found  that  subject  3  had  the  longest  learning  time  of  all  in  tliis  experiment. 
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0    I    2  3   4    5   6   7   8   9  10  11  12  13  14  15  16  17  18  19  20  21  22  23  24  25  26  27  28  29  30 

Sequence  of  Selections 


□   Subject  1       O  Subject 2     □   Subject 3      ■    Subject 4       X  Subjects 


Figux  7.  Individual  Selection  Time  on  Questions  in  the  Menu. 


To  analyze  the  sequence  of  selccling  questions,  Figure  8  indicates  that  most  subjects  just  selected 
questions  in  order  from  the  menu.  All  subjects  accessed  question  16  in  the  same  sequence  and  chose  to  read 
questions  28, 29,  and  30  the  last.  Four  subjects  excepi  subject  3  cho^^e  questions  in  order  after  reading 
question  16. 


Although  all  of  the  questions  were  alphabetically  lisuxi  in  the  menu,  it  was  found  that  non-linear 
selections  occurred  less  often  during  learning.  If  a  person  were  to  make  a  jump  (i.e.,  make  a  non-linear 
selection ),  it  was  likely  that  he  or  she  would  continue  to  read  adjacent  questions  or  resume  the  prior 
sequence  after  the  jump.  For  instance,  subject  1  chose  question  3  to  read  first  and  llien  sequentially  read  the 
next  8  questions,  up  lo  question  11.  Subject  1  jumped  to  read  question  2.  After  reading  question  2,  subject 
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1  chose  question  12,  returning  lo  the  prior  sequence.  Then  subject  1  read  three  questions  (i.e.,  13, 14,  and 
15)  followed  by  question  12.  Later,  subject  1  made  a  jump  at  the  15fh  selection  to  question  1.  However, 
subject  1  returned  to  question  16,  continuing  in  the  prior  sequence  again.  Tho  other  example  was  subject  4 
who  read  questions  10, 1 1 ,  and  12  sequentially  after  reading  the  first  question,  question  9.  Subject  4  then 
jumped  to  question  7.  However,  subject  4  resumed  the  prior  sequence  by  reading  question  13  after  reading 
question  7.  Subject  4  continued  to  read  question  14  and  15.  Then  again  subject  4  made  another  jump  at  the 
12lh  selection  lo  question  6.  This  time,  after  reading  question  6,  subject  4  read  the  adjacent  questions  (i.e., 
5, 4,  and  3)  in  reverse  order. 
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Questions  Listed  in  Menu  In  Order 


□  Subject  1     O  Subject 2        CI   Subject  3        ■    Subject 4         X  Subjects 


Figure  8.  Sequence  of  Questions  Selected  by  OP  Subjects. 


It  was  found  that  only  subject  3  made  many  jumps  while  she  was  selecting  questions  to  read. 
Subject  3  wily  read  questions  4,  5, 6,  and  7,  as  well  as  questions  28,  29,  and  30  sequentially.  As  a  matter 
of  fact,  subject  3  selected  questions  in  a  non-linear  sequence  most  of  the  lime.  In  terms  of  test  scores, 
subject  3  improved  her  pretest  scores  of  4  points  to  1 1  points  in  the  postlcst,  which  was  the  biggest  gain  of 
all  pairs  of  the  prctest-posttest  scores.  Referring  back  to  Figure  7,  one  could  see  that  when  tlie  selection 
time  was  aR)roximately  less  than  10  seconds,  subjects  would  simply  select  questions  in  order. 


67 


1*1  o 

( ^ 


Sclf-Qugsiionins 

As  mentioned  in  the  preceding  section,  the  number  of  qualified  student-generated  questions  on  a 
transfer  test  was  calculated.  Quantitatively,  it  was  found  that  the  same  number  of  qualified  questions  was 
generated  by  the  OP  subjects  and  the  LP  subjects  who  received  questions  in  the  lesson  (see  Figure  9).  The 
LT  subjects  who  received  a  non-question  lesson  generated  rnorc  qualified  questions  than  the  other  iwo 
groups.  To  conclude,  providing  questions  in  the  CBI  lesson  did  not  promote  students  self-questioning 
skills. 
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Figure     Number  of  Qualified  Student-Generated  Questions  on  Postlest  by  Groups. 


Qualitatively,  it  seems  that  students  remembered  the  five  main  idea  questions  provided  by  the 
lesson  such  as  cause,  effects,  measurement,  detection,  and  prone-zones  of  earthquakes.  It  was  found  difficult 
for  students  to  transfer  detailed  questions  specific  to  earthquakes  to  production  of  detailed  questions  regarding 
tornadoes.  Only  few  students  who  knew  more  about  tornadoes  generated  detailed  questions  such  as  "What  is 
in  the  tornado  funnel?"  and  "Why  does  the  warm  and  cold  air  mix  cause  tornadoes?"  In  other  words, 
students  generated  merely  main  idea  questions.  If  a  student  has  more  prior  knowledge  about  a  subject 
matter,  then  he  or  she  may  generate  more  detailed  questions.  In  this  study,  however  students  were  not  tested 
on  prior  knowledge  about  tornadoes.  Upon  examining  student-generated  questions,  it  was  found  that  many 
students  were  concerned  with  the  prediction  and  prevention  of  tornadoes  to  occur,  because  thoy  believed  that 
such  disasters  as  earthquakes  and  tornadoes  could  be  predicted  and  prevented  in  advance. 


CONCLUSION 


This  pilot  study  investigated  the  use  of  questions  in  terms  of  format  and  sequence  in  a  CBI  lesson. 
The  predicted  effect  of  insu-uclional  method  (i.e.,  non-linear  prequestions)  was  not  significant  in  any  of  the 
analyses  possibly  due  to  the  small  sample  size.  Some  interesting  learning  effects  were  found,  however. 
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Alli^ough  students  noted  the  prcquesLions,  the  result  of  this  study  did  not  show  the  benefit  of  open 
accessibility  of  quesiiOTS  (i.e.  non-linear  sequence).  Two  reasons  might  be  offered.  One  is  that  this  lesson 
contained  too  many  questions  for  students  to  select.  Students  may  have  been  more  able  to  fully  utilize  the 
feature  of  open  accessibility  of  questions  if  the  number  of  prequestions  for  selection  had  been  reduced.  It  is 
assumed  that  one  has  a  fixed  amount  of  cognitive  capacity  or  mental  resoufx:es.  Many  options  or  choices 
may  cause  a  learner  to  place  extra  load  demands  on  his  or  her  cognitive  system.  As  a  result,  a  learner  may 
forego  exploring  these  qptlons  or  choices.  At  this  point,  a  learner-controlled  lesson  seems  to  be  no 
different  from  a  lincar-sequenced  lesson  or  a  program-controlled  lesson.  In  fact,  most  students  attempted  to 
make  only  one  or  two  jumps  and  then  read  sequentially  rather  than  to  randomly  choose  questions  every 
single  lime,  although  one  student  with  the  frequent  number  of  non-linear  jumps  improved  the  most  from 
pretest  to  postiest.  Future  research  could  explore  the  reasons  why  students  selected  adjacent  questions  after  a 
jump  and  why  students  jM'eferred  resuming  the  prior  sequence. 

The  outcome  of  a  loanier-coatroUed  instruction  is  also  greatly  influenced  by  individual  differences, 
in  this  study,  some  subjects  were  reluctant  to  freely  Icam  the  lesson  from  the  onset;  some  hesitated  to  take 
control  at  all  stages  of  learning;  while  some  pursued  their  own  unique  ways  throughout  the  lesson. 
Although  these  individual  differences  exist,  a  CBI  lesson  should  motivate  learners  to  maximize  the 
instructional  capability  of  a  lesson  in  the  first  place.  Learners  may  therefore  find  a  better  method  to  learn 
than  the  old  method.  If  this  were  not  to  occur,  a  CBI  lesson  should  also  provide  an  environment  where 
learners  feel  confident  in  any  way  ihcy  prefer  to  learn  and  which  should  be  equally  effective  as  other  methods 
available  in  the  lesson. 
1 
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Interdisciplinary  Multimedia  Learning 
Using  Anchored  Instruction 

Robert  J,  Myers,  Virginia  Polytechnic  Institute  and  State  University 

Abstract 

The  use  of  the  videotape  series,  Voyage  of  the  Mimi,  was  investigated  as  a  vehicle  for 
interdisciplinary  "anchored  instruction."  The  study  centered  around  a  middle  school  class 
of  125  students  taught  by  five  teachers.  It  is  the  position  of  this  paper  that  the  dranmatic 
videotape  series,  Voyage  of  the  Mimi.  creates  an  environment  for  sustained  exploration 
through  relevant  learning  activities.  Data  was  collected  using  observation,  interview, 
analysis  of  student  products,  and  pre  and  post  attitude  surveys.  The  surveys  were 
designed  to  determine  whether  exposure  to  the  videotape  series  would  improve  female 
attitudes  toward  math  and  science.  Results  from  the  two-week  exercise  show  no 
significant  attitude  improvement.  Qualitative  information,  however,  indicates  a 
successful  exercise  full  of  mind-expanding  activities.  The  series  is  popular  with  both  this 
middle  school's  students  and  teachers.  Further  research  is  needed  in  the  teachers' 
evaluation  techniques  of  this  type  of  exercise.  This  series  also  appears  to  be  appropriate 
for  those  researching  inquiry,  discovery,  and  cooperative  learning.  A  surprise  was  the 
extent  to  which  this  series  appears  to  support  student  growth  in  the  domain  of  geography. 

Introduction 

This  paper  reports  the  results  of  a  seventh  grade  teaching  team's  two-week  interdisciplinary 
exercise  based  on  a  video  tape  series.  The  media  employed,  the  Voyage  of  the  Mimi.  is  a  dramatic 
videotape  series  developed  under  the  auspices  of  the  Depanment  of  Education  in  1984;  it  is  distributed 
by  Wings  for  Learning. 

The  Mimi  package  is  designed  to  address  math  and  science.  While  the  tape  series  does  not 
provide  for  interactivity  or  computer  control,  as  most  defmitions  of  multimedia  would,  it  does  provide 
for  the  thematic  integration  of  full  motion  video,  print  materials,  and  computer  progi'ams.  The  series 
consists  of  13,  15-minute  segments  about  a  scientific  expedition  to  study  Humpback  Whales  off  the 
Coast  of  Maine.  The  expedition's  crew  provides  a  tapestry  of  human  interaction.  The  crew  is  headed  by 
a  female  biologist  who  is  awarded  a  grant  to  study  whales.  Among  the  racially,  ethnically,  and  gender- 
mixed  crew,  is  a  crusty  sea  captain,  his  1 1  year  old  grandson,  an  oceanogrcpher,  a  hearing  impaired 
research  assistant,  and  two  teenagers. 

Literature  about  this  highly  popular  and  visible  series  is  limited-especially  research  reports 
concerning  the  its  effectiveness.  The  available  information,  while  anecdotal  in  nature,  generally 
provides  favorable  reviews.  Mastrolia  (1991)  states  that "...  there  is  enough  variety  and  interest  so  that 
any  student  would  become  actively  involved  with  learning..."  (p.  31).  An  article  in  High  Technology 
("Grade  Schoolers",  1985)  cites  a  math  teacher  in  Cambridge,  MA  who  states:  "The  program  is 
stimulating  and  practically  limitless  in  its  usefulness"  (p.  58).  The  latter  article  also  suggests  that  the 
Mimi  series  causes  female  students  "to  view  math  and  science  as  both  fascinating  and  accessible"  (p. 
59).  Manin  (1988)  states  that  tlie  Mimi  materials  "could  create  windows  of  opponunity  in  which 
disruption  in  instructional  routines  occurred  and  could  be  utilized  to  open  a  lesson  up  in  positive  ways" 
(p.  6).  Not  all  reviews,  however,  have  been  favorable. 

In  reviewing  the  Voyage  of  the  Mimi.  Postman  (1985)  states  that  "media  merchants"  have 
created  a  mindless  series,  "as  if  media  themselves  have  no  epistemological  or  political  agenda"  (p.  154). 
He  undermines  a  selling  point  often  found  in  multimedia  literature  by  citing  literature  which  discounts 
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suggestions  that  learning  increases  when  presented  in  a  dramatic  setting.  In  addition,  he  questions  why 
children  should  be  studying  Humpback  Whales,  navigation,  and  map-reading  skills.  He  opines  that  the 
subject  matter  is  highly  inappropriate.  In  short,  he  suggests  that  children  will  only  have  learned  that 
learning  is  a  form  of  entertainment.  Postman's  concerns  raise  some  important,  fundamental  issues 
concerning  epistemology  and  methodology.  Some  of  his  criticisms  will  be  dealt  with  in  later  discussion. 
The  paper  turns  now  to  a  discussion  of  literature  which,  in  this  writer's  view,  supports  Mimi-like 
applications. 

There  are  several  avenues  one  might  explore  in  finding  philosophical,  psychological,  or 
epistemological  suppon  for  a  dramatic  series  like  the  Mimi.  One  avenue  would  be  symbol  systems  and 
dual  channels  of  communication  (Gagn6,  1985;  Kozma,  1987;  Paivio,  1986).  Another  avenue  concerns 
instructional  television  and  its  impact  upon  education-especially  concerning  the  role  that  adult  models 
can  have  on  young  viewers  (Strohmer,  1989).  No  doubt  the  latter  two  avenues  are  worthy  of  further 
discussion,  but  each  deserves  separate  treatment.  This  paper  suggests  that  one  of  the  defining  values  of 
the  Mimi  series  lies  in  the  use  of  "situated  cognition"  or  "anchored  instruction."  Both  of  the  latter  are 
methods  designed  to  provide  life-like  environments  that  show  students  the  importance,  application,  and 
relevance  of  the  subject  domain. 

Gardner  (1991)  and  Bigge  (1976)  remind  their  readers  that  before  formal  schooling,  children 
learn  everyday  tasks  at  a  prodigious  rate-seemingly  with  ease  and  enjoyment.  After  entering  formal 
educational  environments,  however,  many  of  the  same  children  find  learning  difficult-if  not  downright 
boring  and  painful.  Bigge  points  out  that  students  often  fail  to  find  or  understand  the  relevance  of  school 
subjects  to  everyday  life.  The  Cognition  and  Technology  Group  at  Vanderbilt  (CTGV,  1990)  discusses 
Whitehead's  position  that  instead  of  relevance,  schools  promote  the  learning  of  inert  knowledge.  Inert 
knowledge  is  defined  as  that  knowledge  learned,  but  often  forgotten  in  appropriate  situations. 

How  does  decontextualization  affect  the  learning  process?  Brown,  Collins,  and  Dugid  (1989) 
contend  that  students  need  more  than  abstract  concepts  and  canned  examples.  They  suggest  that 
learning  should  be  conducted  in  activity  situated  in  real-life  social,  cultural,  and  physical  contexts.  It 
might  be  impractical,  however,  to  take  every  middle  schooler  on  a  converted  tuna  trawler  to  study  the 
habits  of  Humpback  Whales.  To  get  around  this  problem.  Brown,  Collins,  and  Dugid  suggest  a  concept 
termed  "legitimate  peripheral  participation"  (p.  40).  This  concept  applies  to  persons  not  t?\king  direct 
part  in  an  activity,  yet  still  learning  a  great  deal  from  their  peripheral  position.  With  a  little  imagination, 
this  concept  could  be  likened  to  middle  school  students  viewing  the  videotape  of  the  Mimi's  voyage, 
then  engaging  in  activities  similar  to-or  congruent  with  those  depicted  in  the  series.  Research  by  the 
Cognition  and  Technology  Group  at  Vanderbilt  (CTGV,  1990)  is  a  good  example  of  the  use  of  media 
(videodisc)  to  "situate"  or  "anchor"  the  students  within  a  learning  environment. 

CTGV  used  anchored  (situated)  instruction  to  develop  the  Sherlock  Project.  Their  goals  were  to 
help  develop  rich  mental  models,  and  to  have  students  "learn  to  notice  and  use  relevant  information  to 
prompt  them  to  actively  explore  the  video  and  look  for  clues  to  historical  and  geographic  accuracy"  (p. 
4).  Videodisc  was  their  media  of  choice.  The  authors  opined  that  videodisc  "...allows  a  more  veridical 
representation  of  events  than  text;  it  is  dynamic,  visual  and  spatial;  and  students  can  more  easily  form 
rich  mental  models  of  the  problem  situations"  (p.  3).  Noteworthy  was  the  suggestion  that  this 
environment  was  especially  imponant  for  low-achievers  and  students  with  little  domain  knowledge. 

CTTGV  submits  that  new  infomiation  learned  during  meaningful  activity  is  perceived  as  a  tool 
rather  than  as  inert  information.  They  attempt  to  attack  the  inert  information  problem  with  video 
environments  that  support  sustained  exploration.  According  to  the  authors,  this  sustained  exploration 
allows  views  for  multiple  perspectives  and  permits  opportunities  for  teacher  guided  discovery.  Their 
videos  contain  a  large  variety  of  information  that  allows  students  with  varied  backgrounds  to  find 
something  worthy  of  contributing  to  the  class.  Last,  their  problem  solving  macrocontexts  allow  the 
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students  to  develop  tools  that  form  new,  as  well  as  modify  old  perceptions.  This  is  in  contrast  to 
traditional  instruction  that  emphasizes  mere  facts  to  be  memorized.  It  is  the  position  of  this  paper  that 
the  dramatic  series  in  the  Voyage  of  the  Mimi  videotapes  also  creates  an  environment  for  sustained 
exploration  through  relevant  tool  building  activities. 

Methods 

This  study  was  conducted  during  a  two-week  Voyage  of  the  Mimi  exercise  at  a  middle  school  in 
Blacksburg,  Virginia.  All  core  classes,  throughout  the  school  day,  revolved  around  the  Mimi  theme. 
There  were  125  seventh  grade  students  taught  by  a  veteran  teaching  team.  The  average  years  of  teaching 
experience  for  each  of  four  teachers  was  approximately  twenty.  There  was  also  a  student  teacher  from 
Virginia  Tech  who  played  an  integral  role  in  the  execution  of  the  exercise. 

Although  this  study  was  specifically  interested  in  whether  the  series  supported  an  exploratory 
environment,  a  survey  was  also  designed  to  see  whether  the  perception  and  attitudes  of  female  students 
toward  math  and  science  would  improve.  Results  of  pre  and  post  exercise  attitude  questionnaires  were 
analyzed  using  repeated  measures  analysis  of  variance.  Observation,  evaluation  of  student  products,  and 
pre  and  post  session  interviews  were  also  conducted. 

It  was  the  impression  of  this  writer  that  while  this  teaching  team  was  quite  experienced,  its 
individual  teachers  were  more  comfortable  with  a  traditional  approach  to  instruction.  That  is,  their 
approach  was  more  of  a  behavioral  environment  based  on  traditional  methodology.  In  this  setting,  each 
teacher  served  as  the  expert  and  source  of  information.  Each  reli  *,d  on  behavioral  objectives  and  made  a 
conscientious  effort  to  adhere  to  the  standards  of  learning  outline  I  in  county  and  state  g*  '  lelines. 

It  is  important  to  point  out  the  teachers'  epistemological  and  methodological  backgrounds.  These 
teachers  were  taking  somewhat  of  a  risk  in  using  two  precious  weeks  of  their  spring  semester  to  move 
into  new  and  uncharted  waters.  To  the  observer,  the  teachers  appeared  to  be  excited  and  optimistic^  yet 
they  showed  a  bit  of  uncertainty  about  the  upcoming  adventure.  In  addition,  the  team's  planning  and 
coordination  time  was  limited.  This  onl>  served  to  heighten  the  teachers'  anxiety. 

Mimi  literature  indicated  that  there  was  enough  material  to  support  work  from  two  months  to  a 
year.  This  team,  however,  had  only  two  weeks  to  devote  to  the  exercise.  Obviously,  this  would  be 
somewhat  of  an  abridged  undertaking  and  some  of  the  published  Mimi  modules  could  not  be  used.  One 
of  the  team's  objectives  was  to  determine  whether  or  not  teachers  accustomed  to  using  traditional 
methods  could  work  within  an  envTonment  said  to  be  supportive  of  constructivist  methodologies.  The 
teachers  were  also  investigating  whether  the  Mimi  materials  could  be  effectively  integrated  into  the 
curriculum. 

There  was  a  variety  of  materials  for  use.  The  team  had  the  basic  "Overview"  manual  which 
followed  the  video  tapes.  It  offered  advise  to  teachers,  making  suggestions  for  in-class  activities.  It  was 
generic  in  the  sense  that  it  offered  suggestions  that  applied  to  every  discipline.  There  was  also  an 
interdisciplinary  guide.  The  math  and  science  teacher  had  context-specific  modules  at  their  disposal. 
These  will  be  mentioned  shortly. 

Daily  Events 

The  day  before  each  new  Mimi  segment,  the  team  met  during  a  planning  period  to  view  the  next 
day's  video,  llie  meeting  facilitated  the  coordination  of  activities  and  helped  preclude  duplication  of 
effort.  The  students  spent  the  first,  short,  homeroom  period  viewing  the  day's  Mimi  adventure.  They 
then  left  for  their  classes,  rotating  in  and  out  of  four  classrooms,  The  paper  now  discusses  each  team 
member's  methods  and  content  afforded  by  the  Mimi  series. 
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Math 

The  math  instructor  used  the  Mimi  module  entitled  Maps  and  Navigation.  The  teacher  followed 
the  manual  closely,  having  the  students  perform  exercises  involving,  among  other  things,  map  making, 
iT'jasurement,  distances,  latitudeAongitude,  topographic  maps,  and  triangulation.  Math  problems 
included  ratios  and  time-distance  problems.  The  latter  supported  work  the  students  had  been  studying  in 
their  regular  classes.  During  one  exercise,  the  teacher  went  outside  the  text's  scenario  by  taking  the 
students  outside  the  school  to  estimate  distances  and  determine  azimuths  (direction  in  degrees)  by  use  of 
a  compass.  Both  the  teacher  and  the  students  v/ere  excited  about  this  exercise.  The  instructor  was 
overheard  to  say:  "Golly,  but  it  went  well  today-the  kids  loved  it!"  One  female  student  remarked: 
"This  was  just  like  in  real  life!  It's  something  we  don't  have  to  learn  for  a  test,  then  just  forget!" 

Science 

The  science  instructor  used  the  Mimi  module  Ecosystems.  The  students  studied  food  chains, 
food  webs,  and  scientific  methods.  In  looking  at  scientific  methods,  the  students  compared  methods 
conducted  by  the  Mimi  crew  with  those  conducted  in  the  forestry  industry.  They  also  had  the 
opportunity  to  view  and  comment  on  some  of  the  Mimi  series's  13  videotape  "expeditions"  to  places  in 
which  scientific  activity  was  conducted.  The  students  worked  on  group  projects-conducting  research  in 
the  library,  then  presenting  an  end-of-exercise  food  chain  depicted  on  a  poster  board.  There  was  also  a 
written  exit  project  for  each  student. 

English 

The  English  classes  were  divided  into  those  taught  by  the  regular  teacher  and  those  ta^  ^t  by  the 
student  teacher.  The  former  started  the  students  off  by  exploring  the  relationships  of  the  crev\  embers. 
She  had  the  students  keep  daily  journals-pretending  to  be  an  additional  crew  member.  Classroom 
discussion  included  the  ethics  of  whaling,  as  well  as  how  the  hunting  of  whales  has  changed  over  the 
years.  They  also  explored  methods  of  communication  for  both  humans  and  animals.  Last,  the  students 
got  into  writing  cinquain  and  diamante  forms  of  poetry. 

The  student  teacher  was  fortunate  in  quickly  finding  a  popular  theme.  One  of  the  actors  depicted 
in  the  series  was  hearing  impaired.  Once  the  student  teacher  found  that  some  of  her  students  were  fully 
versed  in  sign  language,  she  and  the  other  students  commenced  to  leam  the  entire  alphabet  in  sign 
language.  Their  signing  culminated  in  an  entire  class  taught  using  signing.  The  students  taught  the 
class-dividing  into  groups  to  help  one  another.  The  observer  was  struck  with  the  patience  and  tenacity 
of  the  student  teaching  groups  during  this  exercise.  Toward  the  end  of  the  Mimi  experience,  the  section 
was  visited  by  a  speech  pathologist.  She  discussed  language  as  well  as  testing  for  hearing  impairment. 
The  student  teacher  culminated  her  exercise  by  playing  "Jeopardy"  using  Mimi  terms  and  vocabulary. 

Social  Studies 

The  social  studies  teacher  had  expressed  her  resei-vations  of  such  a  wide-open  approach  to  the 
clas'TOom,  but  was  soon  struck  by  the  students'  interest  in  various  themes  offered  by  the  Mimi  series. 
Tht  class  conducted  research  on  endangered  species;  Smithsonian  and  National  Geographic  magazines 
were  soon  in  abundance.  Next,  the  class  got  into  paleontology-evaluating  evidence  that  whales  had 
evolved  from  land  mammals.  This  led  to  the  review  of  other  evolution-looking  for  example,  at  leopards 
in  transitory  forms  of  evolution.  Codes  were  explored;  the  class  deciphered  coded  written  messages  and 
took  down  messages  flashed  in  Morse  code.  This  class  also  explored  two  other  major  topics.  First,  the 
formation  of  land/water  configurations  such  as  deltas,  straits,  and  bays.  Second,  the  influencp,  of  Indo- 
European  languages  and  their  various  branches. 
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To  this  observer,  the  Mimi  exercise  was  an  unqualified  success.  Postman  was  right,  the  students 
had  fun.  but  so  did  the  teachers.  One  teacher  remarked  that  disciplinary  problems  were  down  and  that 
the  students  were  so  involved  in  the  exercise  that  they  were  bringing  items  from  home  to  support  the 
theme.  One  female  student  became  a  local  celebrity  when  she  reported  on  visiting  a  huge  marine 
museum  in  California.  Others  were  bringing  periodicals  such  as  the  National  Geographic.  Their  intent 
was  to  share  and  show  off  articles  concerning  whales,  fishing,  and  of  course-ihe  Titanic.  Place  mats 
from  Nantucket  were  soon  gracing  the  walls  along  with  the  Mimi  posters.  The  teacher  remarked  that  the 
Mimi  materials  were  a  wonderful  opportunity  to  go  into  so  many  different  aieas.  Instead  of  being  the 
expert,  she  became  more  of  a  facilitator,  coordinator,  and  mentor  in  guiding  the  students  to  critically 
analyze  the  content. 

Statistical  comparison  of  the  pre  and  post  exercise  atutude  questionnaires  was  less  striking.  The 
questionnaire  was  designed  to  determine  whether  females'  attitudes  toward  math  and  science  had 
improved.  The  post  exercise  questionnaire  was  administered  two  weeks  after  the  series.  Of  21 
questions  using  a  Liken  scale,  none  showed  significant  changes  in  attitude.  It  may  be  of  interest  to  note 
that  the  beginning  attitudes  of  the  females  toward  math  and  science  were  not  significantly  different  than 
the  males.  There  may  be  a  number  of  reasons  for  the  lack  of  difference  between  genders.  First,  many  of 
the  children  in  the  sample  group  came  from  families  whose  parents  are  staff  or  faculty  at  Virginia  Tech; 
others  were  the  children  of  professionals  from  Blacksburg.  It  seems  likely  that  these  students  would  be 
college  bound  and  more  interested  in  math  and  science.  The  shortness  of  this  exercise  may  have  also 
contributed  to  no  significant  changes  in  attitudes.  Anecdotal  written  comments  from  the  students, 
however,  were  extremely  positive.  The  students  were  asked  whether  the  next  seventh  grade  class  would 
profit  from  the  Mimi  series.  Here  are  some  examples  of  their  comments: 

•  "...it's  good  for  us  in  all  subjects  and  1  especially  learned  a  lot  about  marine  biology,  so  I  think 

next  year's  class  should  have  a  chance  to  learn  these  things  too." 

•  "...it  was  fun  to  be  learning  about  all  the  same  things  in  all  classes  and  makes  me  get  more 

involved  with  schoolwork." 

•  "...I  liked  how  all  classes  were  incorporated  with  Mimi." 

•  "...This  was  a  great  experience  for  me.  1  enjoyed  all  the  work  we  did.  even  the  homework." 

•  ",..it  is  educational,  shows  how  important  math  and  science  are;  it's  an  interesting  way  to  do  it..." 

•  ",..it  [helps]  students  open  their  eyes  to  what  other  areas  of  science,  math.  etc.  there  are." 

•  "It  was  different  because  the  Mimi  helped  us  refer  the  things  we  learn  to  real  life. 

•  "I  enjoyed  a  lot  of  things  in  this  but  I  think  that  if  anything  should  stay  the  same  for  sure,  it 

should  be  that  the  sign  language  is  taught  again." 

•  "I  would  just  like  to  thank  the  writers  of  this  program  because  I  had  so  much  fun," 

In  fairness,  there  were  some  negative  comments  (approximately  one  percent).  In  the  days  of 
MTV  it  would  be  hard  to  engage  every  student.  There  will  likely  be  a  small  percentage  of  students 
whose  interests  will  always  be  outside  the  classroom. 

In  addition  to  anecdotal  information,  the  students  turned  in  end-of-exercise  papers  that  resembled 
portfolicis.  An  analysis  of  these  papers  showed  that  the  students  were  wrestling  with  important  concepts. 
Examples  include,  but  are  not  limited  to:  human  interaction  with  the  environment-especially  conceming 
endangered  species;  the  social  dynamics  among  the  ship's  crew  members;  the  nature  of  variables  dealt 
with  by  people  with  physical  impairment;  paleontology;  and  geography. 
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Discussion 

The  series  truly  supports  an  interdisciplinary  approach.  One  of  the  unheralded  benefits  of  the 
series,  however,  may  be  in  the  field  of  geography.  Carstensen  and  Morrill  (1992),  in  an  unpublished 
paper  dealing  with  a  geography  project  at  Virginia  Tech,  discuss  the  shortcomings  of  the  nation^s 
students  in  geography.  They  cite  a  need  for  students  to  have  knowledge  of  the  significance  of  location, 
the  origins  and  development  of  places,  and  the  consequences  of  the  interactions  between  humans  and  the 
natural  environment.  The  knowledge  of  location  and  the  physical  characteristics  of  places  is  addressed 
by  them  in  a  module  named  after  the  sport  of  orienteering.  They  state:  "Orientation  skills  are  defined  as 
the  ability  to  determine  one's  position  and  the  location  of  distant  objects  in  relation  to  that  position,  and 
the  ability  to  use  a  compass  and  map  to  navigate  through  an  unknown  area"  (p.  12).  They  submit  that 
this  module  *'...makes  students  more  aware  of  the  role  of  geography  in  their  daily  spatial  behavior" 
(p.l2). 

The  discussion  in  the  preceding  paragraph  was  introduced  to  address  whether  the  students  in  this 
Mimi  exercise  were  just  having  fun,  or  in  the  best  of  all  worlds,  having  fun  and  learning  at  the  same 
time.  In  the  opinion  of  this  writer,  the  Mimi  materials  were  adept  at  directly  addressing  the  types  of 
knowledge  discussed  by  Carstensen  and  Morrill.  The  students  were  learning  geography  while  they  were 
also  engaged  in  the  other  disciplines.  Readers  may  also  wonder  about  the  post  exercise  opinion  of  the 
teaching  team  that  was  originally  concerned  about  standards  of  learning  and  behavioral  objectives.  The 
reader  can  form  his  or  her  own  opinion.  The  team  plans  to  conduct  the  Mimi  exercise  during  two  weeks 
of  the  coming  fall  semester,  and  the  school  has  purchased  the  series  The  Second  Voyage  of  the  Mimi  for 
a  two  week  exercise  in  the  spring  semester. 

During  a  post-exercise  interview,  each  team  member  discussed  plans  for  expansion  of  and  further 
experimentation  with  his  or  her  methodology  in  implementing  the  Mimi  series.  The  math  instructor 
wants  to  move  from  a  basic  map  and  navigation  course  to  one  that  incorporates  actual  orienteering 
exercises  to  be  held  on  the  school  grounds.  The  other  three  teachers  envision  more  of  a  constructivist 
approach  in  which  cooperative  learning  plays  a  greater  role.  Ability  grouping  will  be  a  thing  of  the  past 
during  the  coming  school  year,  so  cooperative  activities  may  be  more  appropriate.  It  is  this  writer's 
impression  that  now  that  the  teachers  have  a  first  hand  experience  with  the  series,  each  can  expand  the 
content  or  create  supplemental  activities  congruent  with  the  series.  This  will  enable  the  teachers  to 
create  a  closer  fit  between  the  curriculum  and  the  Mimi  series. 

Does  the  dramatic  videotape  series,  Vovagc  of  the  Mimi,  provide  an  effective  vehicle  for 
instruction?  Yes.  Within  the  concept  of  anchored  instruction,  it  allows  the  instructor  to  pursue  nearlv  all 
disciplines.  It  also  allows  the  teacher  the  opportunity  to  take  side  trips  into  interestingly  new  areas  while 
the  students  are  engaged  in  learning  activities. 

One  issue  dealt  with  implicitly  in  this  paper  was  the  teachers'  methods  of  evaluation.  This  is  an 
area  that  deserves  more  study  in  general--not  just  in  the  Mimi  context.  This  team's  teachers  relied  on 
daily  observations,  participation,  homework,  projects,  and  end-of-exercise  papers  as  the  basis  of 
evaluation.  The  latter  may  sound  behavioral  in  nature,  but  there  were  no  tests  and  only  a  few  quizzes. 
The  techniques  listed  were  used  more  in  a  constructivist  approach.  There  were  more  right  answers  than 
usual-students  received  A's  on  papers  that  approached  the  topic  from  completely  different  angles. 

The  Mimi  materials  provide  a  very  rich  opportunity  for  further  research.  Measuring  the  spatial 
skills  of  the  students  who  engage  in  the  map  and  navigation  materials  is  an  example:.  Those  engaged  in 
cooperative,  inquiry,  and  discovery  learning  will  also  find  a  fertile  environment. 
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Last,  was  the  expenditure  of  over  $2  million  by  the  Department  of  Education  to  produce  this 
series  a  waste  of  money?  Not  at  all,  it  was  worth  every  penny,  and  more-just  ask  approximately  125 
students  and  five  teachers  at  this  middle  schooL 
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ABSTRACT 

The  potential  of  computer-based  technology  to  help  improve  our  current  educational  system 
depends  on  the  efforts  made  by  schools  to  incorporate  the  technology  into  daily  activities 
of  instruction.  The  phase  during  which  an  innovation  is  integrated  into  daily  instruction  is 
referred  to  as  implementation.  Historically,  implementation  of  computer-based  technology 
in  schools  (from  elemental^  to  college  level)  has  failed  either  because  it  has  not  been 
properly  carried  out  or  the  technology  has  been  met  with  resistance  in  the  schools*  This 
paper  examines  the  problems  associated  with  technology  Implementation  as  well  as  sources 
of  resistance  to  computer-based  technology  in  school  systems.  Also  discussed  are  general 
approaches  and  specific  strategies  for  ensuring  successful  implementation  of  innovations,  as 
put  forth  in  current  literature.  One  strategy,  employing  the  instructional  technologist  as  an 
implementation  mediator  (change  agent),  is  given  special  attention. 


INTRODUCTION 

Computer  technology  holds  great  promise  for  improving  the  current  educational 
system  in  America.  Positive  changes,  however,  cannot  be  accomplished  unless  the 
technology  is  integrated  into  the  regular  activities  of  instruction  in  schools.  While  the 
diffusion  of  a  technology  to  school  systems  is  a  long,  arduous  process  in  itself,  adoption  of 
an  innovation  does  not  guarantee  its  acceptance  or  success.  Acceptance  is  achieved  when 
the  technology  becomes  a  part  of  a  school's  established  pattern,  that  is,  when  it  becomes 
institutionalized  (Levinson,  1990).  Success  is  achieved  when  the  technology  becomes  more 
than  hardware  or  software,  when  it  becomes  the  well-planned  systematic  arrangement  of 
teaching  and  learning  events  accomplished  through  the  use  of  hardware  and  software 
(Fawson  &  Smellie,  1990). 

The  phase  during  which  an  organization  tries  to  incorporate  technology  into  its  daily 
activities  is  often  referred  to  as  the  "implementation  phase"  (Bishop-Clark  &  Grant,  1991). 
Implementation  is  part  of  the  process  of  technology  diffusion  in  which  an  innovation  is 
brought  into  a  new  organization  to  be  used  by  the  members  of  that  organization,  In  the 
case  of  educatiop'\l  organizations,  the  innovai^on  may  be  new  or  revised  materials,  new 
procedures  or  approaches  to  instruction,  or  the  alteration  of  beliefs  (Fullan,  1982), 
Computer-based  technology,  the  focus  of  my  paper,  seems  to  be  all  of  these  things. 
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MODELS  OF  DIFFUSION  AND  IMPLEMENTATION 

I  examined  two  diffusion  models  to  help  explain  the  process  of  implementation. 
According  to  Abbott  (1967),  implementation  is  the  third  stage  of  technology  diffusion.  It 
follows  the  awareness  stage,  during  which  a  dissatisfaction  with  the  status  quo  arises,  and  the 
search  stage,  during  which  new  procedures,  materials,  etc.  are  proposed  that  might  solve  the 
dissatisfaction.  During  im.plementation,  an  organization's  energies  are  devoted  to  integrating 
into  the  organizational  structure  the  innovation  or  innovations  upon  which  they  have  agreed. 
Abbott  writes  that  the  type  of  innovation  adopted  and  the  extent  to  which  it  is  implemented 
depends  on  the  organization's  openness  to  new  ideas  as  well  as  the  rigidity  of  its  rules 
system.  That  is,  the  more  open  an  organization  is  to  new  ideas  and  the  les*"  rigid  its  rules 
system,  the  more  likely  the  organization  will  adopt  an  innovation  and  implement  it  to  a 
great  extent. 

Knirk  and  Gustafson  (1986)  developed  a  somewhat  different  model  for  technology 
diffusion.  In  this  model  individuals  or  groups  usually  proceed  through  the  following  steps 
in  deciding  whether  to  adopt  new  materials  or  processes: 


1)  Awareness         The  individual/group  becomes  aware  that  a  new 

technology  exists. 

2)  Interest  The  individual/group  develops  a  curiosity  about  how  the 

new  material/process  works  and  what  its  benefits  may 
be. 

3)  Appraisal         Individual/group    considers    the    advantages  and 

disadvantages  of  the  technology. 

4)  wdoption  Individual/group  decides  to  make  use  of  the  innovation 

and  integrates  it  into  the  decision  and  behavior  patterns 
of  the  individual  or  group. 


Implementation  is  not  specifically  identified,  but  it  is  included  in  the  adoption  phase  of 
Knirk  and  Gustafson's  model. 


Unlike  Abbott,  these  authors  believe  that,  in  education,  the  individual  not 
organizations  make  the  decision  to  use  a  new  technology.  Also,  the  focus  is  not  so  much 
on  a  dissatisfaction  with  the  status  quo  but  by  educators'  curiosity  about  how  an  innovation 
could  improve  the  current  system.  Knirk  and  Gustafson  do  seem  to  agree  with  Abbott  on 
a  few  of  the  factors  which  influence  the  diffusion  of  innovations  in  schools  or  other 
organizations  -  that  is,  the  characteristics  of  the  organization  and  the  characteristics  and 
attitudes  of  the  adopter.  In  this  list  of  influential  factors,  Knirk  and  Gustafson  would 
include  the  characteristics  of  the  innovation  and  the  adopter's  understanding  of  the 
innovation. 
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Besides  examining  two  models  of  diffusion,  I  also  looked  at  a  model  of 
implementation  put  forth  by  Levinson  (1990).  Levinson  breaks  down  the  implementation 
process  into  three  distinct  phases  -  planning,  installation,  and  institutionalization.  During 
planning,  individuals  identify  what  problems  will  be  solved  and  design  the  architecture  of 
the  computer-based  system  (i.e.  how  the  computer  functions  at  the  classroom,  school,  and 
district  levels,  as  well  as  how  the  computer  equipment  will  be  linked  throughout  these 
levels).  The  planning  phase  aiso  includes  building  coalitions  because,  according  to  Levinson, 
"change  needs  its  alliances  for  support."  The  second  phase,  installation,  requires  that  the 
school  and  the  technology  undergo  changes.  The  role  and  function  of  the  technology  is 
more  clearly  defined  (or  redefined),  and  teachers  learn  skills  necessary  to  manage  the  new 
technology.  Teachers  also  will  assess  and  redefine  their  roles  in  their  classrooms  because 
the  innovation  will  change  the  way  instruction  has  been  delivered  in  the  past.  The  final 
phase,  institutionalization,  completes  the  process  of  implementation.  The  aim  is  to  maintain 
and  support  the  new  system  as  efficiently  as  possible.  Until  it  becomes  a  part  of  the 
structure  of  the  school  system  and  the  daily  activities  of  instruction,  it  is  at  risk  of  failing. 

The  models  described  above  all  have  their  merits,  but  none  is  comprehensive  in 
describing  the  processes  of  technology  diffusion  and  implementation  in  schools.  I  propose 
an  expanded  model  which  combines  the  ideas  of  Abbott,  Knirk  and  Gustafson,  and  Levinson 
into  a  six-stage  process.  Before  I  talk  about  the  expanded  model,  I  want  to  comment  on  the 
locus  of  decision.  I  have  explained  that  Knirk  and  Gustafson  stated  that  their  approach  was 
based  on  the  idea  that  individuals,  not  organizations,  make  decisions  about  adopting  a  new 
technology,  and  that  Abbott's  focus  was  on  the  organization's  implementing  an  innovation. 
My  approach  flip-flops  the  researchers'  ideas,  I  believe  that,  in  most  cases,  organizations 
(in  the  form  of  boards,  committees  and  organizational  leaders,  each  of  whom  is  elected, 
appointed,  formed  or  hired  to  make  decisions  for  the  organization)  will  make  the  decisions 
about  whether  or  not  to  adopt  a  new  technology.  However,  individuals  make  the  decision 
about  whether  to  implement  the  new  procedure,  materials,  or  belief  systems.  Organizations 
can  adopt  an  innovation  and  either  recommend  or  force  its  use  on  the  members,  of  the 
organization,  but  that  does  not  guarantee  its  implementation.  Individuals  are  ultimately 
responsible  for  the  integration  and  validation  of  a  technology.  In  schools,  the  teachers  make 
the  decisions  concerning  implementation  of  computer-based  technology,  according  to  Hativa 
(1986). 

Below  are  listed  the  six  phases  of  diffusion  with  brief  explanations: 

1.  Awareness         There  is  a  dissatisfaction  with  the  status  quo  or  a  feeling 

that  improvements  could  be  made. 

2.  Search  Individuals  within  an  organization   learn  of  new 

technologies  or  innovations  that  might  solve  the  problem 
or  bring  about  improvements. 
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3.  Appraisal 


Individuals  or  groups  weigh  the  advantage  and 
disadvantages  of  each  innovation  for  their  system.  If 
they  find  one  that  is  suitable,  they  recommend  it  to  the 
decision  makers. 


4.  Adoption 


The  decision  makers  decide  to  adopt  a  new  technology 
and  introduce  it  into  the  organization. 


5.  Implementation 


The  technology  is  integrated  into  the  structure  and 
behavior  patterns  of  the  organizations.  This  involves 
planning  the  architecture  of  the  new  system,  installing 
the  technology,  evaluating  it,  adapting  it  if  necessary,  and 
institutionalizing  it. 


6.  Continuation  Individuals  within  the  organization  and  the  organization 
itself  support  and  maintain  the  new  system,  adapting  it 
when  necessary  to  meet  future  changes. 

The  above  model  is  helpful  in  understanding  how  technologies  go  from  the  "factory" 
(designer/developer)  to  the  "customer"  (schools  and  teachers).  Also,  this  model  could  be 
expanded  to  take  into  account  the  mind-set  of  the  designer/developer  of  the  innovation,  but 
for  the  purposes  of  this  paper,  this  framework  is  sufficient. 

PROBLEMS  WITH  IMPLEMENTING  NEW  TECHNOLOGIES  IN  SCHOOLS 

Abbott  (1967)  writes  that  the  school  is  a  social  system  whose  function  is  considered 
in  terms  of  a  continuous  series  of  interactions  and  transactions  between  it  and  its  supporting 
environment.  To  perform  its  functions,  and  thus  maintain  a  steady  state  with  its 
environment,  the  school  receives  inputs  in  the  form  of  personnel  and  material  resources, 
uses  these  inputs  as  sources  of  energy,  and  produces  (as  output  to  the  environment)  students 
who  have  been  subjected  to  its  influence.  Thus,  schools  are  undergoing  and  producing 
changes  everyday. 

Change  can  stabilize  an  environment,  but  it  also  can  upset  its  balance  such  that 
adaptation  requires  major  modifications  in  the  structure,  behavior,  or  function  of  an 
environment.  Change  is  usually  uncertain,  often  expensive,  and  almost  always  time- 
consuming.  Fullan  (1982)  claims  that  the  source  of  change  in  education  is  not  always 
focused  on  improving  the  organization.  That  is,  people  may  implement  change  for  the 
wrong  reasons  (e.g.  self-aggrandizement  or  securing  resources  the  organization  could  not  get 
any  other  way).  When  discussing  schools,  we  often  remark  that  change  is  a  good  thing.  In 
many  instances,  change  is  good,  but  in  other  matters  we  must  recognize  that  change  is  not 
always  a  positive  thing. 
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Implementing  computer-based  technology  into  school  systems  represents  the  type  of 
change  that  upsets  social  systems  like  schools.  According  to  Levinson  (1990),  "if  technology- 
mediated  education  is  to  transform  our  schools,  we  must  change  the  way  we  implement 
change."  Not  only  will  schools  have  to  make  changes  in  the  delivery  of  instruction  using 
computers,  they  also  will  have  to  change  the  ways  in  which  they  incorporate  computer-based 
technologies  and  other  innovations  into  their  organizations.  Computer-based  technology 
affects  the  way  in  which  all  members  of  an  organization  behave  -  from  the  individual  who 
makes  the  budget  decisions  to  the  person  responsible  for  the  security  of  the  equipment  in 
schools. 

Because  computer-based  technologies  are  information  technologies  -  that  is,  the 
material  that  "runs"  computers  is  information  -  they  seem  to  have  been  more  difficult  to 
integrate  into  schools  than  perhaps  other  physical  technologies.  Information  is  nonphysical 
and,  therefore,  nonhomogeneous,  so  the  applications  of  information  technologies  are  much 
less  straightforwardly  predictable  and  their  net  effect  is  hard  to  quantify  (Aksoy,  1991). 
Schools  lack  precise  information  about  the  effectiveness  of  computers  in  improving 
instruction.  There  is  no  consensus  about  just  how  much  computers  enhance  instruction  in 
schools,  and  given  the  expense  of  computer  equipment,  schools  may  be  reluctant  to  allocate 
a  larger  portion  of  their  resources  for  computer-based  technologies. 

Allum  (1991)  argues  that  "implementation  is  not  a  short-term,  one-time  event,"  but 
rather  a  continuous  process  involving  long-range  planning,  continued  commitment  from 
teachers  and  administrators,  and  a  systematic  program  of  evaluation.  As  well,  effective 
educational  change  depends  more  on  understanding  of  what  a  particular  school  or  teacher 
needs  from  an  innovation  rather  than  dictating  how  a  school  should  proceed  in  utilizing  the 
innovation  or  evaluating  how  well  an  organization  masters  the  innovation.  To  help  ensure 
the  successful  implementation  of  computers  in  education,  the  processes  of  teaching  and 
learning  must  be  considered  (Ennals,  1987). 

Levinson  (1990)  identified  the  following  components  of  effective  change  in  schools 
attempting  to  integrate  computers  into  instructional  activities:  (1)  identifying  the  critical 
problem  to  be  solved  before  purchasing  new  equipment;  (2)  considering^  stakeholders  in 
deciding  how  the  computer-based  technology  will  be  used  in  schools,  (3)  reaching  a 
consensus  about  how  it  will  be  used;  (4)  assuming  new  roles  for  teachers  and  administrators, 
practicing  new  patterns  of  behavior,  and  learning  new  skills;  and  (5)  actually  integrating  the 
computer-based  technology  into  the  daily  activities  of  the  school  organization.  Creamer  and 
Creamer  (1988)  also  identified  forces  that  influence  change  within  an  institution.  (See 
Appendix  A.) 

BARRIERS  TO  IMPLEMENTATION 

Given  this  discussion  about  the  uncertainty  and  difficulty  of  change  within  school 
systems,  one  would  expect  there  to  be  barriers  or  resistance  to  the  implementation  of 
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computer-based  technology*  Rose  (1982)  grouped  the  sources  of  resistance  into  four 
categories:  institutional  economic  barriers,  technological  barriers,  administrative  barriers, 
and  educator  barriers.  These  barriers  were  applied  to  the  current  situations  in  primary  and 
secondary  schools,  but  they  also  have  been  observed  in  institutions  of  higher  education 
(Click,  1990). 

The  institutional  economic  barriers  include  a  lack  of  funding  to  buy  and  maintain 
computer  equipment  and  a  lack  of  human  and  nonhuman  resources  committed  to  the 
implementation  of  the  technology.  In  a  typical  school  system,  almost  90  percent  of  the 
budget  is  taken  up  by  labor,  debt  service,  and  other  fbced  costs,  so  instructional  technology 
is  often  seen  as  a  "budgetary  frill"  (Foshay,  1988).  A  vicious  circle  seems  to  have  developed 
in  terms  of  the  financial  issue:  Schools  will  not  spend  money  on  a  technology  that  is  not  a 
part  of  the  organization's  structure,  and  technology  will  not  become  a  part  of  the  school's 
structure  unless  resources  are  allocated  to  its  integration.  Foshay  also  says  that  technology 
often  has  been  brought  into  schools  and  then  expected  to  take  root  and  flourish 
spontaneously.  Change  cannot  take  place  unless  it  is  managed  carefully  and  supported  by 
those  who  are  managing  the  change. 

There  also  are  technological  barriers  to  the  implementation  of  innovation  in  schools. 
First,  teachers  find  it  bothersome  to  adapt  or  change  course  content  and  teaching  styles  to 
the  technology.  If  they  have  methods  that  work,  they  do  not  want  to  make  changes  to  them. 
Furthermore,  there  is  no  predicting  that  using  the  computer  will  improve  what  procedures 
or  materials  are  already  \a  place.  A  second  technological  barrier  is  that  instructional 
technologies  are  perceived  only  as  tools  for  teachers,  not  as  integrated  alternatives  to 
current  instruction.  While  it  is  the  nature  of  the  computer  as  a  tool  that  may  keep 
computers  in  schools,  schools  will  not  benefit  from  computers  unless  the  technology  becomes 
an  integral  part  of  the  teaching  and  learning  processes  (Collins,  1991).  Other  technological 
barriers  are  that  the  equipment  may  not  be  available  or  accessible  to  teachers  when  they 
wish  to  use  it  and  that  computer  technologies  may  not  be  perceived  as  useful  in  meeting 
instructional  needs  or  improving  instructional  delivery. 

The  human  element,  in  the  form  of  administrators  and  educators,  are  also  sources 
of  resistance  as  identified  by  Rose.  Administrators  may  cause  teachers  to  dislike  the  new 
technology  because  they  force  the  teachers  to  use  it  in  order  to  justify  the  high  cost  of  the 
equipment.  And  while  administrators  advocate  the  use  of  computer  technology  in  the 
classroom,  they  do  not  support,  appreciate,  or  reward  teachers  who  use  it.  Compounding 
this  problem  is  the  fact  that  many  schools  have  not  developed  formal  procedures  for 
evaluating  the  results  of  computer  use  in  their  classrooms.  The  absence  of  formal  formative 
evaluation  procedures  makes  it  difficult  for  administrators  to  know  how  well  computers 
improve  instruction,  to  decide  how  the  computer-based  system  should  be  maintained  or 
improved,  or  to  determine  who  should  be  rewarded  for  effectively  integrating  the  computer 
into  the  classroom.  Lastly,  institutional  leaders  are  failing  to  lead  their  schools  during  the 
crucial  implementation  phase.  Unless  the  administrators  champion  the  implementation  of 
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computer  technology  and  make  it  a  priority  during  budget  planning,  schools  will  not  benefit 
from  the  power  of  computer  technology  to  improve  instruction. 

The  final  source  of  resistance  to  innovation,  according  to  Rose,  is  that  created  by 
educators.  First,  educators  may  be  unaware  of  new  technologies  or  they  do  not  care  to 
learn  about  them.  It  may  be  that  they  know  about  them  but  do  not  understand  how  to  use 
them.  Teachers  may  perceive  the  computer  as  a  threat  to  their  jobs.  They  are  concerned 
that  computers  will  replace  them  or  that  computers  will  force  them  to  give  up  their  control 
over  learning  and  instruction.  And  even  if  they  are  willing  to  try  a  new  instructional 
approach,  they  may  become  disillusioned  with  the  notion  of  using  the  alternative  method 
because  they  do  not  receive  support  for  their  efforts.  Teachers  also  worry  about  the  ability 
of  the  equipment  and  software  to  perform  properly  during  instruction  the  failure  of  the 
equipment  being  a  potentially  embarrassing  situation.  Finally,  educators  do  not  have  the 
time  to  design  and  develop  quality  computer-based  instruction  that  meets  their  specific 
needs.  A  small  piece  of  courseware  can  take  many  hours  to  develop  -  time  teachers  do  not 
have  to  spare. 

Gross,  Giacquinta,  and  Bernstein  (1970)  claim  that  most  researchers  attribute  the 
success  or  failure  of  planned  organizational  change  to  the  organization's  openness  or 
resistance  to  change,  respectively.  They  studied  a  school  attempting  to  implement  a  major 
innovation  to  which  members  had  an  initial  positive  orientation  and  found  that  this  school 
also  encountered  barriers  to  implementation.  These  barriers  are  listed  as  follows: 

1)  Lack  of  clarity  on  the  part  of  organizational  members  about  the  innovation 
and  what  is  expected  in  terms  of  implementation 

2)  Lack  of  the  capabilities  necessary  to  carry  out  the  innovation  on  the  part 
of  the  members 

3)  Lack  of  tools  or  equipment  to  carry  out  the  innovation 

4)  Incompatibility  of  the  existing  organizational  conditions  with  the 
implementation  of  the  innovation 

5)  Permission  and  incentives  not  given  to  subordinates  to  carry  out 
implementation 

Implementation  is  difficult  even  when  schools  open  their  arms  to  innovations  like  computer- 
based  technology. 

STRATEGIES  FOR  FACILITATING  IMPLEMENTATION 

A  number  of  authors  have  proposed  strategies  or  guidelines  for  overcoming  resistance 
to  innovations  in  schools  (Rose,  1982;  Foshay,  1988;  Dalton,  1989;  Fawson  &  Smellie,  1990; 
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Click,  1990;  Levinson,  1990;  Persky,  1990;  Bishop-Clark  &  Grant,  1991).  Some  of  the 
general  guidelines  are  as  follows: 

-  Build  cooperative  computer-based  learning  environments 

-  Develop  integrated  software  that  supports  routine  curriculum  objectives 

-  Provide  teacher  training  in  the  use  of  innovations 

•    Let  the  individual  teacher  decide  whether  or  not  to  use  the  innovation 

-  Define  new  roles  for  teachers  (facilitator,  developer,  and  manager) 

-  Increase  the  monetary  and  personnel  resources  available  for  change 

-  Reward  and  support  the  users  of  nontraditional  instructional  approaches 

-  Develop  procedures  for  evaluating  the  effectiveness  of  instructional 
technologies 

-  Engage  an  instructional  technologist  (teacher  or  outside  consultant)  as  a 
change  agent,  someone  who  assists  in  the  implementation  process 

A  more  general  approach  to  the  successful  application  of  educational  technologies 
in  schools  is  put  forth  by  Foshay  (1988).  Instructional  technology  (in  this  case,  computer- 
based  technology)  should  be  defined  as  a  systematic  and  comprehensive  methodology  rather 
than  as  hardware  or  software.  Schools  and  instructional  technologists/change  agents  must 
develop  strategies  for  introducing  technologies  into  schools  as  part  of  a  comprehensive 
instructional  improvement  effort.  Computers  should  no  longer  be  isolated  from  other 
innovative  efforts  to  improve  the  educational  system. 

Foshay  also  states  that  instructional  technology  should  be  applied  in  "fertile  ground," 
organizations  that  are  committed  to  implementing  computers  into  the  school  system  as  well 
as  the  financial  means  *o  do  so.  These  schools  are  models  for  those  who  do  not  have  a 
clear  commitment  or  plan  for  technology  im.plenTentation  in  their  system. 

I  would  add  to  this  general  approach  the  recommendation  for  building  a  consensus 
among  organizational  members  about  how  to  implement  the  technology  in  that  particular 
school  (Levinson,  1990;  Allum,  1991).  Also,  instructional  technologists  as  change  agents 
must  do  what  Tessmer  and  Harris  (1990)  call  an  "environmental  analysis"  -  the  analysis  of 
the  physical  and  instructional  characteristics  of  product  use  in  order  to  describe  how  the 
instruction  will  be  employed  in  the  real  world.  All  the  strategies  for  facilitating 
implementation  must  be  carried  out  not  only  by  teachers  and  school  administrators  but  also 
by  the  educational  technologists  who  wish  to  bring  new  innovations  to  school  systems. 

THE  INSTRUCTIONAL  TECHNOLOGIST  AS  CHANGE  AGENT 

When  change  happens,  people  respond  to  it  by  adapting  to  it,  ignoring  it,  or  making 
use  of  it.  In  order  for  schools  to  benefit  from  large-scale  changes  like  computer-based 
instruction,  they  must  carefully  manage  the  processes  of  implementation  and  continuation, 
"School  systems  need  reliable  assistance  in  the  processes  of  integration,  assessment,  and 
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public  relation  contact),  his/her  role  as  "change  agent"  may  be  the  most  important  in 
relation  to  schools.  The  basic  role  of  a  change  agent  is  to  determine  strategies  that  will 
result  in  the  adoption,  acceptance,  and  integration  of  a  new  educational  technology  (Knirk 
&  Gustafson,  1986).  Change  agents  often  are  third  parties  outside  the  school  organization 
who  can  provide  technical  and  implementation  expertise  without  having  direct  control  or 
power  over  the  intended  users  (Bishop-Clark  &  Grant,  1991).  Although  the  IT  is  somewhat 
limited  in  the  number  of  implementation  strategies  he/she  can  choose  in  a  particular  setting, 
the  IT  may  have  a  greater  chance  of  success  than  a  teacher  or  administrator  because  the 
technologist  is  less  of  a  threat. 

Dalton  (1989)  suggests  that  instructional  technologists  are  in  a  unique  position  to 
manage  the  process  of  change  within  school  systems.  "What  distinguishes  educational 
technology  as  a  discipline  is  the  systematic  or  scientific  methodology  that  we  apply  to 
practical  problem  in  education.  We  analyze,  design,  develop,  and  evaluate  solutions."  He 
adds  that  the  most  significant  role  of  the  change  agent  is  to  act  as  an  interface  between  the 
adopters  of  an  innovation  and  those  individuals  with  a  vested  interest  in  seeing  change 
occur.  However,  the  most  frequent  mistake  made  by  ITs  as  change  agents  is  that  they  fail 
to  examine  their  solutions  to  educational  problems  in  light  of  the  needs  of  the  intended 
implementors:  teachers  (Dalton,  1989).  We  must  remember  that  teachers  ultimately  make 
the  decision  about  whether  or  not  an  educational  innovation  will  be  implemented.  Change 
agents  must  keep  teachers  in  mind  at  all  times  when  designing  strategies  for  the 
implementation  of  computer-based  technologies  in  schools. 

Knirk  and  Gustafson  (1986)  have  identified  a  list  of  skills  that  are  positively 
correlated  with  effective  change  agents: 

-  Is  client'  and  need-centered  rather  than  material-  or  innovation-centered 

-  Is  somewhat  more  educated  than  the  client 

-  Is  perceived  as  knowledgeable  and  competent  by  the  client 

-  Is  open  to  information  from  a  variety  of  sources 

-  Works  hard  to  communicate  information  about  the  technology  to  intended 
users 

-  Seeks,  empathizes  with,  and  gets  the  support  of  leaders  in  the  adopting 
organization 

Miles,  Saxl,  and  Lieberman  (1988)  also  developed  a  list  of  18  skills  that  are  key  for  effective 
change  agents.  Among  the  key  skills  are  relating  simply  and  directly  to  others,  developing 
a  nurturant,  supportive  relationship  with  the  client,  facilitating  team  won:,  orchestrating  sliC 
implementation  process,  and  resolving  or  improving  situations  where  incompatible  interests 
are  in  play.  (For  a  complete  list,  see  Appendix  B.) 

The  skills  mentioned  above  make  it  easier  for  instructional  technologists  to  effect 
change  within  the  educational  settings  in  which  they  are  based.  The  skills  aiso  help  the 
technologist  in  carrying  out  the  following  tasks  of  implementation  mediation,  as  identified 
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by  Bishop-Clark  and  Grant  (1991):  (1)  understanding  the  sources  of  resistance  to  innovation 
within  an  organization;  (2)  educating  and  communicating  with  teachers  about  the  new 
technology  and  about  the  implementation  process;  (3)  encouraging  positive  attitudes  toward 
technology;  (4)  ialroducing  change  that  is  consistent  with  the  social  and  educational 
environs  ..^nt  of  the  school;  (5)  supporting  and  encouraging  the  champions  in  schools,  those 
who  actively  and  v,nthusiastically  promote  a  technology. 

CONCLUSION 

Because  there  is  little  need  to  develop  a  product  or  technology  that  will  not  be  used, 
understanding  the  process  of  technology  implementation  becomes  very  important  for  schools 
and  for  instructional  technologists  acting  as  change  agents  in  those  schools.  It  is  also 
important  to  recognize  that  change  is  not  always  good  for  schools,  despite  all  the  current 
problems  in  our  educational  system.  The  basic  approach  to  implementing  change  in  schools 
should  be  a  focus  on  the  needs  and  desires  of  the  implementors  and  intended  users  of  new 
technologies  or  materials.  Schools  should  recognize  and  take  advantage  of  the  potential  of 
instructional  technologists  to  implement  change  effectively.  In  turn,  the  instructional 
technologist  must  recognize  and  confront  the  possible  barriers  to  implementation  as  well  as 
determine  strategies  to  encourage  the  integration  of  educational  technologies  into  the  daily 
activities  of  schools. 


90 


11 


REFERENCES 


Abbott,  M.G.  (1967).  The  school  as  a  social  system:  Indicators  for  change.  In  J.V. 
Baldridge  &  T.E.  Deal  (Ed.)  Managing  change  in  educational  organizations 
(pp.  176-186),  McCutchen  Publishing  Co.:  Berkeley,  CA. 

Aksoy,  A.  (1991).  Computers  are  not  dynamos:  Frontiers  in  the  diffusion  of  information 
technology.  Futures,  23  (4),  402-414. 

Allum,  K.F.  (1991).  Partners  in  innovation:  School-college  collaborations. 
EDUCOM  Review,  26  (3/4),  29-33. 

Bishop-Clark,  C.  &  Grant,  R.  (1991).  Implementing  computer  technology  in  educational 
settings.  Journal  of  Educational  Technology  Systems,  19  (4),  313-326. 

Collins,  A.  (1991).  The  role  of  computer  technology  in  restructuring  schools.  Phi  Delta 
Kappan,  7J  (1),  28-36. 

Creamer,  E.G.  &  Creamer,  D.G.  (1988).  Predicting  successful  organizational  change: 
Case  studies.  Journal  of  College  Student  Development,  29  (1),  4-18. 

Dalton,  D.W.  (1989).  Computers  in  the  schools:  A  diffusion/adoption  perspective. 
Educational  Technology,  29  (11),  20-27. 

Ennals,  R.  (1987).  Difficulties  in  managing  innovation.  British  Journal  of  Educational 
Technology,  IS  (3),  194-198. 

Fawson,  E.C  &  Smellie,  D.C  (1990).  Technology  transfer:  A  model  for  public 
education.     Educational  Technology,  30  (4),  19-25. 

Foshay,  W.R.  (1988).  Educational  technology  in  schools  and  in  business:  A  personal 
experience.  Educational  Horizons,  66  (4),  154-157. 

Fullan,  M.  (1982).  The  meaning  of  educational  change.  Teachers  College  Press: 
New  York. 

Glick,  M.D.  (1990).  Integrating  computing  into  higher  education.  EDUCOM  Review, 
25  (2),  35-38. 


12 


Gross,  N.,  Giacquinta,  J.B.,  &  Bernstein,  M.  (1975).  Failure  to  implement  a  major 

organizational  innovation.  In  J.V,  Baldridge  &  T,E.  Deal  (Ed,)  Managing  change 
in  educational  organizations  (pp.  409-426),  McQitchen  Publishing  Co.: 
Berkeley,  CA. 

Hativa,  N,  (1986).  Computer  guided  teaching:  The  micro-computer  revolution.  Journal 
of  Educational  Computing  Research,  2  (3),  307-325. 

Knirk,  F.G.  &  Gustafson,  K.L.  (1986).  Instructional  technology:  A  systematic  approach 
to  education.  Holt,  Rinehart  and  Winston:  New  York. 

Levinson,  E,  (1990).  Will  technology  transform  education  or  will  the  schools  co-opt 
technology?  Phi  Delta  Kappan,  72(2),  12M26. 

Miles,  M.B.,  Saxl,  E.R.,  &  Lieberman,  A.  (1988).  What  skills  do  educational  "change 
agents"  need?  An  empirical  view.  Curriculum  Inquiry,  18  (2),  157-93. 

Persky,  S.E.  (1990).  What  contributes  to  teacher  development  in  technology. 
Educational  Technology,  30  (4),  34-38. 

Rose,  S,N.  (1982).  Barriers  to  the  use  of  educational  technologies  and  recommendations 
to  promote  and  increase  their  use.  Educational  Technology,  22  (12),  12-15. 

Tessmer,  M.  &  Harris,  D.  (1990).  Beyond  instructional  effectiveness:  Key 

environmental  decisions  for  instructional  designers  as  change  agents.  Educational 
Technology,  30  (7),  16-20, 


92 

102 


APPENDIX  A 


CREAMER  &  CREAMER'S  KEY  CONDH}   .  IS  FOR  CHANGE 


Variable 

Condition 

Circumstances 

The  presence  of  a  uniform  perception  of  a  need  for 
change 

Value  compatibility 

The  project  ar.d  its  plan  for  implementation  are  seen 
as  useful  and  harmonious  with  other  procedures 

Idea  comprehensibility 

The  project  goals  and  ways  to  implement  them  are 
articulated  clearly 

Practicality 

Adequate  personnel  and  resources  are  sustained 
throughout  the  planning  and  implementation  of  the 
project 

Top-level  support 

Top-level  leadership  exhibits  sustained  commitment  to 
the  project 

Championship 

A  clearly  definable  project  champion  serves  as  an 
influential  advocate  empowered  with  the  responsibility 
to  implement  project  goals 

Leadership 

A  clearly  identifiable  project  leader  maintains 
commitment  and  support  throughout  the  planning  and 
early  implementation  of  the  project 

Advantage  probability 

Outcomes  of  the  project  are  apparent  and  perceived 
to  address  significant  institutional  concerns 

Strategies 

A  distinction  is  maintained  between  the  process  of 
conceptualizing  the  fundamental  focus  of  the  project 
and  the  process 
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APPENDIX  B 


MILES,  SAXL,  &  LIEBERMAN'S  KEY  SKILLS  FOR  CHANGE  AGENTS 


Sklii 

Description 

Interpersonal  ease 

Relating  simply  and  directly  lo  others 

Group  functioning 

Able  lo  facilitate  teamwork 

Training/Doing  workshops 

Directs  instruction;  teaches  adults  in  systematic  way 

Educational  general/ 
master  teacher 

Has  wide  educational  experience;  able  to  impart  skills  to  others 

Educational  content 

Has  knowledge  of  school  subject  matter 

Administrative/ 
organizational 

Defines  and  structures  work,  activities,  and  time 

Initiative-taking 

Starts  or  pushes  activities;  moves  directly  toward  action 

Trust/rapport«building 

Develops  a  sense  of  safety,  openness,  and  reduced  threat  on  the 
part  of  clients;  good  relationship-building  abilities 

Support 

Provides  nurturant  relationships  with  and  among  clients 

Confrontation 

Able  to  express  negative  information;  can  challenge  what  works 
and  what  does  not  without  generating  a  negative  effect 

Conflict  mediation 

Resolves  or  improves  situations  where  multiple  incompatible 
interests  are  in  play 

Collaboration 

Creates  relationships  where  influence  is  mutually  shared 

Confidence-building 

Strengthens  clients'  sense  of  self-efficacy  and  belief  in  self 

Diagnosing  individual's  problems 

Forms  a  valid  picture  of  the  needs  and  problems  of  an  individual 
teacher  or  administrator  as  a  basis  for  action 

Diagnosing  organization's  problems 

Forms  a  valid  picture  of  the  needs  and  problems  of  a  school  as  an 
organization  (including  its  culture)  as  a  basis  for  action 

Managing/controlling 

Orchestrates  the  change  process;  coordinates  activities,  time,  and 
people;  directly  influences  others 

Resource-bringing 

Locales  and  provides  information,  material,  practices  and 
equipment  that  is  useful  to  the  client 

Demonstration 

Can  demonstrate  the  desired  new  behavior  while  in  meetings  and 
in  the  classroom 
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Designing  Computer^Suppoited  Intentional  Learning  Environments 

Joanna  C.  Dunlap  and  R.  Scott  Grabinger,  University  of  Colorado 

Computer- supported  intentional  learning  environments  (CSILE's)  provide  students  vith  a  forum 
for  subject  matter/content  area  exploration  based  on  individual  needs  and  interests.  Using  a 
CSILE  as  an  instructional  tool,  designers  and  teachers  enable  students  to  take  responsibility  for 
and  control  of  their  learning  goals  and  processes.  CSILEs  provide  for  student-directed 
knowledge  base  creation  through  collaborative  involvement  utilizing  higher  order  questioning, 
elaboration,  productive/constructive  commenting,  and  learning  process  management  skills. 
Before  student  contributions  are  added  to  a  CSILE  system,  CSILE  is  simply  a  computer-based 
shell.  Designers  must  consider  five  main  issues: 

-What  is  the  nature  of  the  subject  matter/content  that  will  be  entered  by  the  students 

during  knowledge  base  creation? 
-How  will  students  want  to  express  their  contributions  to  the  knowledge  base  (e.g., 

textually,  graphically,  audibly)? 
-How  will  students  want  to  retrieve  the  knowledge  contained  in  the  database? 
•What  is  the  appropriate  level  of  control  to  incorporate  into  the  structure  of  the  shell 

(e.g.,  linking  capabilities  available)? 
-How  will  teachers/instructors  want  to  incorporate  CSILE's  into  the  curricula,  and  how 

should  they  be  incorporated? 

This  presentation  demonstrated  how  session  participants  can  create  computer-supported 
intentional  learning  environments. 


Electronic  Performance  Support  Systems  Demonstrated  to  Reduce  Cholesterol 

Among  Users! 

Robert  C.  Fratini,  Al  <kT  National  Product  Training  Center 

Electronic  performance  support  systems  (EPSS)  have  burst  on  to  the  human  performance  scene  within 
the  past  several  years.  As  with  such  previous  "revolutionary"  concepts  as  computer-based  training,  the 
vanguard  espousing  the  development  of  EPSS  tend  lo  claims  of  new  paradigms  and  the  utter  bankruptcy 
of  the  performance  technology  which  preceded  EPSS. 

Exactly  how  revolutionary  are  EPSS?  Are  we  better  off  forgetting  everything  that  we  ever  learned  about 
the  design  of  user  documentation  and  interactive  instruction  or  are  EPSS  just  the  latest  tool  in  a 
performance  technologist's  toolkit?  In  what  situations  can  EPSS  be  developed  cost  effectively  given 
today's  technology?  And  what  are  realistic  expectations  of  what  EPSS  can  and  cannot  do  to  enhance 
human  performance? 

The  presentation  examines  EPSS  in  the  context  of  an  overall  human  performance  support  system  which 
consists  of  multiple  components,  each  addressing  different  aspects  in  the  enabling  of  performance.  To 
the  extent  that  EPSS  do  represent  a  new  paradigm,  performance  technologists  must  be  capable  of  shifting 
their  perspectives  to  incorporate  elements  of  EPSS  where  appropriate. 
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Evaluating  M indtoois  in  Schools 

David  Ji.  Jonassen,  University  of  Colorado 


Mindtools  are  computer  application  tools,  such  as  database  management  systems,  spread  sheets, 
semantic  networking  programs,  hypertext,  expert  systems,  and  microworlds,  such  as  Logo  and 
Bubble  Dialog  that  may  be  used  by  students  to  analyze  content  domains  across  the  curricula  as 
well  as  reflect  their  own  knowledge  structures  which  engage  them  in  cognitive  learning 
strategies.  140  junior  high  school  students  spent  four  months  during  the  spring  building  content 
databases  in  Works,  creating  semantic  networks  using  Learning  Tool,  and  creating  expert 
systems  rule  bases  in  Primex. 

No  significant  differences  in  gains  on  the  critical  thinking  test  were  found,  however,  the 
treatment  period  was  shorter  than  desired.  Fluid  intelligence  predicted  performance  on  all 
scales.  Both  student  and  teacher  reactions  to  the  project  were  very  enthusiastic. 


Exploration  and  Experience  with  Second  Generation  Authoring 

Paul  J.  Magelli,  Software  to  Educate  People,  Inc. 

This  session  will  begin  by  reviewing  the  problems  and  requirements  for  second  generation 
authoring  systems.  The  development  of  courseware  with  the  current  generation  of  authoring 
technology  is  slow  and  produces  courseware  that  is  difficult  and  expensive  to  manage  and 
maintain.  These  systems  collapse  the  learning  dimensions  of  instructional  material  -  such  as  its 
organization  and  instructional  strategy  -  and  forces  the  instructional  developer  to  develop 
courseware  as  a  sequence  oi  frames. 

This  talk  will  focus  on  an  exploration  of  second  generation  authoring  technology  in  commercial 
courseware  development  and  the  experience  gained  with  it.  This  technology  offers  the  ability 
to  capture  knowledge  in  a  knowledge  base  and  supports  the  automatic  construction  and 
reconfiguration  of  courseware  with  different  content,  organization,  instructional  strategy,  and 
media.  It  also  supports  the  reuse  of  previously  captured  knowledge  and  the  ability  to  easily  and 
quickly  re-target  instructional  material.  The  talk  will  present  the  experience  gained  including 
a  reduction  in  courseware  development  time,  maintenance  time  and  improved  courseware 
quality.  The  presentation  will  also  look  at  the  challenges  ahead  for  this  new  technology. 
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Building  Sophisticated  Learning  Softv^^are  in  MuStiniedia 

John     Whitaker,  Arthur  Andersen  &  Co. ,  Societe  Coopertive 

Andersen  Consulting,  working  in  cooperation  with  Northwestern  University's  Institute  for  the 
Leamin}  Sciences  created  the  Business  Practices  Course  (BPC).  BPC  simulates  a  client 
engagen-  t  through  15  modules  at  Perrin  Printing  &  Publishing,  a  case  company.  Using  over 
180  minutes  of  audio  and  video  segments  stored  on  CD-ROM,  the  learner  conducts  interview 
with  PP&P  personnel,  receives  phone  calls,  reviews  memos  from  Andersen  Consulting 
supervisors,  and  even  attend  meetings  with  PP&P  upper  management.  As  with  actual  client 
engagements,  the  course  allows  for  multiple  solutions  to  most  activities. 
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The  40-hour  BPC  replaces  approximately  65  hours  of  self-study  and  40  hours  of  instructor-led 
training.  Over  3,500  Andersen  Consulting  personnel  worldwide  will  complete  BPC  each  year. 
Estimated  cost  savings  are  about  8  million  dollars  per  year  which  is  comprised  of  housing, 
tuition,  travel  and  training  reduction  costs. 

This  session  provides  participants  with  valuable  insight  into  developing/converting  courses  into 
a  highly  interactive  self-study  format  utilizing  multimedia.  Through  a  short  lecture  and 
demonstration,  the  participants  will  gain  a  first  hand  look  at  the  development  of  this  complex 
CBT  project. 


A  Computer-Based  Simulation  of  Planned  Change 

Michael  Yacci,  Rochester  Institute  of  Technology 

The  Simulation  of  Planned  Change  is  a  computer  simulation  that  allows  a  student  to  experience 
the  effects  of  various  change  strategies  in  a  social  system.  The  simulation  is  specifically  based 
on  the  ideas  and  models  of  change  described  by  Rogers,  Havelock,  and  Zaltman  and  Duncan. 
In  this  simulation,  the  student  is  asked  to  make  a  series  of  decisions  regarding  innovation  and 
change;  the  computer  then  dynamically  reacts  to  students  choices. 

The  Macintosh-based  simulation  is  framed  as  an  interactive  game  in  which  a  single  learner 
interacts  with  the  computer.  The  learner  plays  the  role  of  the  Minister  of  Diffusion  in  a  fictional 
country  which  is  under  financial  pressure  to  modernize  its  industrial  base.  The  modernization 
options,  the  strategies  used  to  implement  these  options,  and  the  results  of  these  decisions  are  the 
essence  of  the  game. 

This  presentation  outlines  the  final  version  of  the  simulation  and  also  discusses  the  process  by 
which  it  was  created.  Finally,  the  instructional  uses  for  the  simulation  will  be  discussed. 
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Advisor  Development  Life  Cycie 

Marie-Mich&le  Boulei,  Ph.D. 
Full  Professor 
Deparment  of  Computer  Science 
Faculty  of  Science  and  Engineering 
University  Laval 
Quebec,  CANADA  G1K7P4 
Email  Internet;  Boulet@vml.ulavaJ.ca 

Abstract  -  A  development  life  cycle 
pertaining  to  intelligent  advisor  systems  is 
presented.  Its  main  attribute  is  to  be  user  oriented. 
That  means  the  better  set  of  tactics  for  recalling 
concepts,  principles,  and  rules  pertaining  to  a 
complex  transfer  task  is  first  identified.  After, 
decisions  about  physical  implementation  are  taken. 
Technical  aspects  do  not  have  any  influence  on  the 
strategy  of  intervention.  Extracts  of  MUSIC 
documentation,  a  musical  composition  intelligent 
advisor,  illustrate  each  phase  of  the  development 
life  cycle. 

Introduction 

An  intelligent  advisor  system  is  "an  adaptive 
system  aiming  at  intervening  when  the  user  of  any 
given  software  performs  transfer  tasks"  (Boulet, 
1992).  A  transfer  task  "consists  of  principles  (or 
guidelines)  that  an  expert  uses  to  generate  the 
appropriate  performance  for  any  given  situation.  It 
also  consists  of  secondary  content,  such  as 
concepts  and  information"  (Leshin  et  al.,  1992). 
Designing  and  developing  an  intelligent  advisor 
system,  claiming  to  be  able  to  help  a  user 
performing  a  transfer  task,  is  a  very  complex 
undertaking.  It  is  to  plan  a  help  that  surrounds  an 
individual,  aiming  at  furthering  the  transfer  of 
several  knowledge  previously  learned. 

To  design  such  a  surrounding,  the  best  set  of 
tactics  for  presenting  to  software  users  concepts, 
principles,  and  rules,  being  part  of  a  complex 
transfer  task,  must  be  identified.  This  set  of  tactics 
aims  at  favoring  the  emergence  of  the  best  solution. 
To  favor  it,  this  instructional  technology  assists 
both  the  thinking  process  of  an  individual  facing  a 
complex  problem,  the  planning  process  of  the 
appropriate  solution,  the  process  of  carrying  out  the 
plan,  and  the  process  of  validating  the  solution. 
While  the  advisor  is  helping,  it  collects  data  to  be 
able  to  adapt  the  content  of  its  help  transactions  to 
the  user,  and  to  propose  a  relevant  remedial  help. 


An  intelligent  advisor  system  is  a  technology 
to  be  used  after  initial  learning  of  concepts, 
principles,  and  rules.  An  intelligent  advisor  does 
not  aim  at  replacing  the  formal  teaching;  it 
addresses  the  transfer  of  knowledge.  A 
development  life  cycle  tliat  considers  requirements 
for  this  particular  instructional  technology  from  a 
user  standpoint  is  described. 

Description  of  the  steps 

Step  1.  Describing  the  su-atcgy  of  intervention 

1.1  Stating  the  aim  of  the  advisor 

What  the  advisor  will  bring  to  the  user  is 
defined:  MUSIC  aims  at  individualizing  feedback 
when  a  grade  9  high  school  student  performs 
musical  creation  activities  described  in  the  Qudbcc 
Minisu-y  of  Education  music  curriculum  (Minisifere 
de  Education,  1981,  1983). 

1.2  Characterizing  the  target  users 

The  target  users  of  MUSIC  are  grade  9  high 
school  students  with  weak  to  satisfactory 
knowledge  of  the  musical  language  elements  listed 
in  the  Quebec  Ministry  of  Education's  music 
curriculum,  and  liulc  to  no  experience  transferring 
it. 

1 .3  Describing  the  help  transactions 

An  advisor  is  designed  to  help  someone: 
MUSIC  does  a  human  teacher  cannot  do,  i.e., 
performing  concurrently  several  tasks.  There  are 
thirty  learners,  and  the  lesson  lasts  fifty  minutes.  It 
is  impossible  for  a  single  teacher  to  give 
concurrently  fast,  individualized,  and  detailed 
explanations  and  feedbacks  on  each  learner's 
composition,  and  to  recall  prerequisites;  MUSIC 
does. 

1.4  Describing  the  main  characteristics  of  the 
course  of  interventions 

While  the  user  composes,  and  in  accordance 
with  his  or  her  request,  MUSIC  answers 
questions,  comments  on  the  user's  composition, 
creates  its  own  compositions,  and  comments  on 
them. 
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1.5  Describing  how  the  advisor  adapts  ilsclf  to  the 
user 

There  are  several  possibilities.  Examples  arc 
presented  in  the  following.  The  advisor  can  engage 
the  user  in  a  dialogue;  according  ^o  the  user's 
feedback,  it  will  modify  the  content  of  its  next 
interveniions.  It  can  be  decided  to  grade  the  user. 
She  or  he  can  select  among  levels  displayed,  or  a 
grading  test  can  be  used.  Inferring  the  user 
knowledge  level  is  another  possibility.  The  nature 
of  requests  can  be  analyzed.  That  corresponds  to 
answer  to  the  following  question:  Do  the  user 
requests  refer  to  prwequisite  knowledge?  The 
Elaboration  theory  (Reigeluth,1987;  Rcigeluth  et 
al.,  1983)  can  facilitate  the  identification  of 
prerequisite  knowledge  to  a  complex  if ansfer  task. 
Prerequisite  knowledge  being  identified,  ii  is 
possible  to  link  user  requests  to  a  level  within  a 
hierarchy;  this  level  can  after  be  used  to  determine 
the  user  one.  Another  v/ay  is  to  produce  a  list  of 
systematic  errors  with  their  causes;  models  coming 
from  the  field  of  measurement  and  evaluation  are 
useful.  A  user's  question  (or  a  set)  can  be  related  to 
a  particular  error.  One  can  decide  to  divide  the 
domain  in  regard  of  prerequisite  knowledge,  and  to 
link  systematic  errors  to  the  corresponding 
performance  objective.  The  user  knowledge  level 
can  also  be  inferred  using  the  frequency  of  requests 
for  help,  or  errors,  or  both,  relatively  to  a  same 
topic.  The  main  problem  is  to  define  the  norm:  how 
many  requests  or  errors  are  necessary  to  infer  that  it 
is  enough.  MUSIC  provides  individuali/.cd 
feedback  and  help  by  considering  student's 
requests  and  his  or  her  composition.  MUSIC  refers 
to  prerequisite  knowledge  (i.e.,  rules  issued  from 
tonal  system,  and  from  the  musical  composition 
processes  (such  as  augmentation,  diminution, 
question-answer,  repetition)),  when  answering  the 
student's  requests,  commenting  on  the  student's 
composition,  composing  its  own  melody,  and 
commenting  on  it. 

1.6  Overall  characteristics  of  the  subject  of 
interveniions 

Examples  of  questions  to  be  answered  are: 
Will  the  advisor  comment  on  the  problem  solving 
process,  or  the  results?  The  objective  slated  in  the 
Quebec  Ministry  of  Education  music  curriculum 
(1981,  1983)  is  not  to  learn  the  right  way  to  .solve 
the  problem  of  composing  a  melody. 


Consequently,  MUSIC  will  solely  comment 
on  results,  i.e.,  the  melody  composed.  The  main 
objective  being  to  transfer  the  knowledge  of 
musical  language  elements,  MUSIC  outlines 
prerequisite  knowledge  within  each  intervention. 

1 .7  Overall  characteristics  of  vocabulary  of 
presentation 

As  an  advisor  aims  at  helping  a  user  while 
performing  a  transfer  task,  the  short  term  memory 
is  the  main  concern;  means  to  allow  the  user  to 
recall  prerequisite  knowledge  are  planned. 
Examples  are:  using  demonstrations,  examples, 
generalities,  index,  maps,  pictures,  or  procedures. 
Leshin's  and  a).  (1992)  design  strategies  and  tactics 
can  be  referred  to.  MUSIC  does  three  types  of 
interventions:  1-  explanations  requested  by  a 
student  in  regard  of  musical  language  elements,  2- 
comments  on  compositions  made  up  by  the 
advisor,  3-  comments  on  the  student's 
composition.  Explanations  present  either  a 
generality,  or  a  statement  of  principle  or  rule;  when 
suitable,  an  example  is  displayed.  Melodies 
composed  by  MUSIC  are  in  accordance  with  the 
musical  composition  process  currently  used:  for 
example,  the  composition  of  a  G  Major  melody, 
eight  bars,  using  the  question-answer  process,  time 

signature  being  Ay  range  of  the  melody  of  an  octave. 
Comments  on  the  student's  composition,  or  on  its 
own,  is  done  taking  into  account  the  way  the 
musical  language  elements  are  used. 

1.8  Interface 

The  interface  is  described:  natural  language, 
menu  driven,  buttons,  keywords,  etc.  MUSlCs 
interface  is  a  mixed  one.  Buttons  and  menus  allow 
the  selection  of  a  particular  option.  Keywords 
refers  to  a  given  explanation.  Natural  language  is 
used  to  comment  on  a  composition.  Commenls  are 
pre-stored  sentences  having  variables.  Content  of 
those  variables  is  adjusted  considering  the 
characteristics  of  the  composition.  Mechanisms 
related  to  gender  and  number  are  installed. 

At  the  next  step,  designers  focus  on  strategy 
and  means  to  acquire  knowledge.  It  is  instructional 
design  oriented  so,  ihcy  don't  make  any  a  priori 
choice  about  encoding  the  knowledge  in  the  advisor 
data  su-uclure  as  for  "black  box"  (e.g.  issue-based 
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tutoring,  Burton,  et  al.,  1982),  and  "glass  box 
model"  (e.g.  knowledge-engineering  techniques, 
Clancey,  1982)  approaches. 

Step  2.  Describing  the  knowledge  acquisition 
strategy 

2. 1  Boundary  of  the  domain 

Means  to  delimit  the  domain  are  described. 
That  corresponds  to  analyze  and  characterize  the 
transfer  task. 

2.2  To  identify  problems 

Means  to  identify  problems  an  individual 
may  have,  when  he  or  she  performs  a  complex 
transfer  task  X,  while  using  an  application  software 
Y,  are  selected  and  described.  To  identify  the 
knowledge  usually  not  mastered,  a  standardized 
test  can  be  used.  The  proportion  of  target  users  that 
the  advisor  will  fail  to  help  because  some  of  their 
questions  are  not  part  of  the  test  can  be  estimated. 
Then,  the  number  and  cost  of  errors  that  users 
could  make  despite  the  investment  in  the 
technology  can  be  estimated.  It  can  be  decided  to 
analyze  how  a  representative  sample  of  target  users 
performs  the  transfer  task.  The  performance  can  be 
observed,  or  shoo  ted.  These  users  state  their 
questions  out  load,  designers  list  ihcm.  Because 
irrelevant  variables  can  affect  the  environment,  it 
must  be  rigorously  surrounded.  The  proportion  of 
users  the  advisor  will  fail  to  help  will  be  estimated, 
using  values  such  as  users  not  represented  in  the 
sample;  the  cost  of  the  advisor's  failure,  utility  of 
the  advisor,"etc.  will  be  estimated  too.  ll  can  be 
decided  to  perform  a  sim.ulation  and  an  analysis  of 
protocol.  Means  ensuring  the  sample  actually 
covers  the  variety  of  future  users  must  be  planned. 
Means  ensuring  the  (subjective)  process  of  linking 
a  request  for  help  to  a  category  will  be  objectively 
done  have  to  be  planned.  To  avoid  a  loo  large  lost 
of  information  (simplification  of  the  reality),  there 
must  be  an  appropriate  number  of  categories.  Here 
again,  proportion  of  users  that  are  not  represented 
in  the  sample,  reliability  and  validity  of  the  case 
study  with  error  associated,  etc.  will  be  used  to 
estimate  the  proportion  of  future  users  that  will  not 
be  helped  by  the  advisor  (because  their  questions 
were  not  collected  so,  the  advisor  is  not  able  to 
answer);  the  linked  risk,  i.e.,  they  make  mistakes, 
will  be  estimated. 


2.3  To  identify  sources  to  write  interventions 

Means  to  write  content  of  interventions  the 
advisor  does,  when  an  individual  performing  a 
complex  transfer  task  X,  while  using  an  application 
software  Y,  makes  a  request  for  help  or  mistake, 
r:e  then  selected  and  described.  There  are  several 
possibilities:  specialized  manuals,  course  syllabi 
from  academic  or  professional  sector,  content  of 
courses  given  in  the  academic  or  professional 
sector.  Will  courses  be  observed,  or  shooted?  Will 
experts  doing  the  task  and  expressing  out  load  how 
they  perform  the  complex  task  be  observed,  or 
shooted?  Will  a  simulation  of  the  advisor  be  setted 
and  followed  by  an  analysis  of  protocol  allowing 
the  identification  of  experts'  answers  when  facing  a 
user's  request?  In  any  case,  cautions  mentioned 
before  in  regard  of  the  users'  requests  for  help 
identification  and  categorization  are  relevant.  Those 
mentioned  in  regard  of  calculating  values  to  be  used 
to  estimate  the  margin  of  error  of  the  advisor  (i.e., 
what  proportion  of  users  the  advisor  will  fail  to 
help)  are  also  relevant.  Decisions  on  how  data 
collected  in  regard  of  explanations  given  by  experts 
will  be  processed  must  be  taken.  Analysis  of  these 
data  must  allow  to  identify  explanations  that  really 
helped  the  user  to  perform  the  complex  transfer 
task.  Then,  the  proportion  of  users  that  will  not  be 
helped  even  if  the  advisor  is  able  to  answer  their 
precise  question  will  be  estimated.  Risk  of  errors 
users  can  make  with  cost  related  wiil  be  estimated 
too. 

To  determine  what  will  be  the  content  of 
interventions,  it  can  be  decided  to  refer  to  a  certain 
form  recommended  by  researchers.  Those 
rcvsearchers  recommend  it  after  having  done  several 
rigorously  controlled  experiments  involving  several 
samples;  these  experiments  were  done  to  identify 
how  to  facilitate  the  mastery  of  a  particular  capacity 
described  by  an  action  verb.  As  example,  we 
mention  instructional  tactics  proposed  by  Leshin,  et 
al.  (1992)  in  regard  of  each  type  of  capacity  they 
define.  The  use  of  those  principles  or  tactics  can 
contribute  to  minimize  the  margin  of  error. 
Moreover,  the  values  necessary  to  estimate  the 
utility  and  the  risk  of  failure  can  be  found  in  the 
literature  related.  The  following  comes  from 
MUSIC  documentation. 
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Objectives  stated  in  the  high  school  music 
curriculum  relate  to  a  spiral  curriculum.  To  make 
easier  the  delimitation  of  the  domain,  the 
Manhattanville  Music  Curriculum  Program  (s.d.) 
will  be  consulted. 

Previous  step  documents  mention  that  access 
to  prerequisites,  and  to  links  between  them,  will  be 
ensured  by  outlining  keywords  within  each 
explanation.  We  recommand  that  a  network  be 
elaborated  in  order  to  illustrate  what  concepts, 
principles,  and  rules  are  referred  to,  when  a 
particular  explanation  is  displayed. 

MUSIC  has  to  be  able  to  comment  on  each 
student's  composition.  It  also  has  to  be  able  to 
compose  a  melody,  and  to  comment  on  it.  In  regard 
of  rules  to  analyze  or  compose  a  melody,  it  is 
recommended  to  define  determinant  elements:  those 
that  must  be  enforced,  and  those  that  can  be  used. 
An  example  of  mandatory  determinant  element  (as 
far  as  the  question-answer  musical  composition 
process  is  concerned)  is:  the  number  of  bars  of  the 
answer  must  be  equal  to  the  question  one.  The 
following  is  an  example  of  determinant  element  that 
can  be  used:  the  relation  between  the  question  and 
the  answer  can  be  parallel,  semi-parallel,  or 
contrasting.  To  write  i  ervcntions  related  to 
prerequisites,  it  is  recommended  to  refer  to  the 
Qu(5bec  Ministry  of  Education's  documentation,  the 
Manhattanville  Music  Curriculum  Program,  and 
specialized  books.  To  identify  determinant 
elements,  it  is  recommended  to  have  musical 
composition  activities  performed  on  paper  by 
students.  Music  teachers  and  professional 
composers  will  then  comment  on  these  melodies. 
Weaknesses  will  be  listed;  it  will  also  provide  many 
samples  of  comments  done  by  experts  to  help  each 
student  making  progress.  It  is  also  recommended  to 
refer  to  specialized  books  addressing  the  topic  of 
musical  composition  to  list  the  determinant 
elements.  These  elements  being  used,  the  advisor 
will  be  able  to  compose,  and  to  comment  either  on 
its  own  composition,  or  on  the  student's  one. 

2.4  To  set  out  the  knowledge  acquisition  method 


The  following  summarizes  the  knowledge 
acquisition  method  proposed  for  MUSIC, 

1.  Identification  of  Uie  prerequisites 

2.  Specialized  books  analysis  (Musical  language  in 
general) 

3.  Synthesis  in  view  of  writing  a  tentative  content 

4.  Links  between  prerequisite  knowledge 

5.  Students'  compositions,  and  music  teachers  and 
professional  composers'  comments  analysis 

6.  Specialized  books  analysis  (Musical 
composition) 

7.  Synthesis  to  list  determinant  elements  and 
identify  comments  related 

8.  Sequencing  the  activities  in  accordance  with  the 
spinil  cuiTiculum 

To  summarize,  at  this  step,  sources  that  will 
be  used  to  delimii  the  domain  are  identified.  Then, 
means  to  collect  knowledge  (problems,  and 
interventions)  are  selected.  Means  to  assess  quality 
of  this  knowledge  are  proposed:  characteristics  of 
the  sample,  means  of  control  when  categorizing, 
usefulness  of  explanations,  etc.  Means  to  evaluate 
margin  of  error  of  the  advisor  are  also  selected. 
Doing  so,  it  makes  easier  to  evaluate  if  it  is  worth 
to  invest  in  this  particular  technology.Step  3. 
Description  of  the  content,  the  architecture,  and  the 
strategy  of  implcmenlalion  in  the  environment 

3. 1  To  analyze  sources  of  knowledge 

Sources  of  knowledge  identified  at  the 
previous  step  are  analyzed  in  accordance  with  the 
knowledge  acquisition  method. 

3.2  To  write  the  content  of  interventions 

Each  intervention  is  written  in  accordance 
with  the  knowledge  acquisition  method.  If  there  are 
persons  that  will  write  any  part  (think  about  an 
automatic  collecting  of  real  cases,  or  an  automatic 
collecting  of  exercises),  its  structure  will  be 
carefully  detailed.  Doing  so,  the  style  will  be  kept 
unvarying. 


Having  identified  sources  of  knowledge  and 
means,  steps  of  the  corresponding  knowledge 
acquisition  method  arc  then  detailed.  Note  iho'  l^c 
method  is  not  a  generic  one. 
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3.3  To  produce  the  archiiecLure  of  the  advisor 

The  container  being  considered,  the  advisor 
structure  is  outlined,  without  taking  any  physical 
aspects  into  account.  Global  characteristics  of  the 
advisor  are  illustrated  and  documented.  The  main 
components  are  identified.  The  identification  of 
these  main  components  brings  about  the 
identification  of  the  main  modules,  and  of  links 
between  them. 

To  elaborate  models  of  data  and  treamients, 
conventional  techniques  such  as  structured  analysis 
for  treatments  (data  flow  diagrams),  and  conceptual 
database  modelling  for  data  (entity-relationship 
diagram),  or  object  oriented  analysis  can  be  used. 

3.4  To  document  the  architecture 

While  producing  the  architecture,  analysts 
have  to  document  it  using  a  data  dictionnary. 

3.5  To  describe  the  strategy  of  implementation  in 
the  environment 

Works  done  focused  on  one  aspect  of  the 
problem,  being  the  transmission  of  the  knowledge 
in  a  help  context.  Modelling  tasks  currently  done 
focus  on  another  aspect,  being  to  find  the  best 
technological  solution  for  the  transmission  of 
knowledge  in  regard  of  this  particular  context. 
Simultaneously,  works  related  to  the  problem  of 
implementing  an  instructional  technology  in  the 
environment  begin. 

The  strategy  of  implementation  of  the  advisor 
in  its  environment  is  defined  to  make  sure  that  all 
the  target  users  will  always  have  the  information 
they  need,  no  matter  changes  occurred  within  the 
organization.  An  example  is  an  employee  that  quits; 
it  is  not  sure  that  the  information  transmitted  in 
order  to  prepare  the  environment  to  the  installation 
of  any  given  instructional  technology,  i.e.  aim, 
type  of  help,  limits,  etc.,  will  be  available  to  the 
new  person  hired.  To  be  sure  that  everybody 
always  have  all  the  information  needed  to  use  the 
advisor,  or  always  have  an  easy  and  quick  access 
to  that  information,  whatever  changes  occur  in  the 
organization,  means  to  have  a  detailed  plan  of 
communication  for  all  advisor  life  long.  It  also 
means  that  someone  must  be  permanently  and 
actually  affected  to  the  management  of  the  post 
implementation. 


Step  4.  Description  of  the  physical  implementation 

4. 1  To  convert  logical  models  into  physical  models 

At  the  previous  step,  designers  did  logical 
modelling.  They  now  have  to  convert  it  into  a 
physical  model.  It  can  be  decided  to  prototype.The 
following  is  an  example  of  reason  behind  the 
decision  to  prototype  a  part  of  MUSIC:  "Domain 
experts  elaborated  at  the  previous  step,  a  first 
version  of  explanations.  ...  The  prototype  being 
developed,  content  and  form  of  explanations 
changed.  At  first,  domain  experts  wrote  a  quite 
literary  content.  ...  using  the  prototype,  they 
realized  that  this  definition  was  not  very  helpful.  In 
the  same  way,  the  prototype  was  used  by  a  sample 
of  future  students  it  was  asked  to  each  student 
to  state  in  his  or  her  own  words  what  was  the 
meaning  of  the  explanation  displayed."  (Boulet, 
1992). 

4.2  To  validate  the  content  of  interventions 

Prototypes  allow  testing  parts  with  the  target 
users  before  the  whole  advisor  be  programmed. 
Adjusimenis  are  easier  and  cheaper  than  later. 

4.3  To  program 

Program  the  advisor  is  the  point  at  which 
application  programs  arc  written  or  purchased. 

4.4  To  test  the  programs 

To  test  the  programs,  methods  proposed  in 
the  literature  aie  used. 

4.5  To  implement  the  advisor 

Programs  are  run,  the  different  files  are 
interlaced,  and  the  human-interlace  is  setted. 

Functioning  of  the  advisor 

As  illusuated  at  figure  I,  when  a  student  calls 
the  advisor,  he  or  she  can  select  the  process  of 
composition. 
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Process  of  composition 
□  Answer  (Q-A) 

E| Augmentation,  diminution,  repetition 

EjQuestion  -  Answer 

ElQuit 


0  J  g 

^1 

FIG  1  The  main  menu. 

Figure  2  illustrates  a  student  asking  an 
explanation  about  the  accidental  concept.  Some 
words  are  outlined.  If  the  student  selects  one  of 
them,  he  or  she  will  have  the  explanation  related. 


1  Functions  I    Piano  t" 


Accidental) 
It  is  a  sign  aiming  at  elevating  o 
reducing  of  a  semi  tone  the  note  it 
is  in  front  of.  There  are  f lat . 
natural .  and  filiaxi). 


R?^c  (qui  r  I 
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FIG  2  Answering  to  a  student's  request. 


Conclusion 

The  life  cycle  is  user  oriented.  That  means 
designers  don't  have  in  mind  any  a  priori  technical 
solution  such  as  "To  use  HyperCard",  "To  develop 
a  natural  language  interface",  or  "To  use  an 
interactive  videodisc  system";  that  also  means  they 
don't  make  any  a  priori  choice  about  encoding  the 
knowledge  in  the  advisor  data  structure.  The  first 
three  steps  of  the  development  life  cycle  emphasize 
on  the  creative  thinking  process  related  to  the 
identification  of  all  the  details  of  the  strategy  of 
intervention,  the  user  being  considered.  Time  and 
again,  the  focus  is  on  how  the  intelligent  advisor 
will  help  the  user  that  performs  a  complex  transfer 
task.  The  system  analysis  and  models  associated 
come  after. 
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Cognitive  Maps  Before  and  After  Using 
a  Knowledge  Construction  Environment 


Joanna  C.  Dunlap 
R.  Scott  Grabinger 


Abstracfc 


Discovering  the  effectiveness  of  collaborative  learning  environments  using  recall  nieasures  is  a  fairly  basic 
task.  However,  examining  the  effect  of  those  environments  on  thought  processes  is  much  more  difficult  because 
it  involves  inferential  methods.  One  of  these  methods  is  the  use  of  maps  representing  the  perceived 
relationships  between  several  pairs  of  concepts  associated  with  a  specific  topic  using  pathfinder  or 
multidimensional  scaling  techniques.  This  study  exanrdned  the  effect  of  working  within  a  computer  supported 
learning  environment  on  the  participants'  knowledge  structures.  The  findings  indicated  that  after  working 
//ith  the  learning  environment  the  participants*  knowledge  structures  became  more  complex  and  sophisticated. 


In  toda/s  complex  and  tmcertain  environment,  simply  knowing  how  to  use  tools  for  gathering  information  is  not 
enough  to  remain  competitive.  Industry  specialists  are  reporting  fervently  that  people  at  every  organizational 
level  must  now  be  highly  creative  and  flexible  problem  solvers  —  an  ability  based  on  knowledge  construction 
skills  and  not  simply  information  gathering  skills.  Even  members  of  the  so-called  "blue  collar''  workforce  are 
now  required  to  demonstrate  an  advanced  level  of  problem  solving  skill  in  order  to  attain  and  letain 
employment  (e.g.,  employees  working  on  computerized/robotic  assembly  lines  need  to  be  able  to  troubleshoot 
problems  involving  highly  technical  and  advanced  machinery  which  requires  the  ability  to  flexibly  use  past 
experience  and  knowledge  to  address  new  problems).  Consequently,  learning  to  think  critically  and  to  analyze 
and  synthesize  information  (construct  knowledge)  to  solve  technical,  social,  economic,  political,  and  scientific 
problems  is  crucial  to  a  modern,  competitive  society.  New  technologies  involving  multimedia  and  hypermedia 
applications  have  increased  our  ability  to  create  learning  environments  that  help  teach  people  to  achieve 
these  goals. 

However,  in  our  increasingly  diverse  society ,  it  is  more  and  more  difficult  to  give  people  the  individual 
attention  they  need  to  develop  the  necessary  critical  thinking  and  problem  solving  skills.  Learning  these  skills 
is  no  longer  something  that  can  take  place  in  an  isolated  setting  —  such  as  within  one  course  or  a  six-week  time 
frame.  Instead,  reasoning  and  problem  solving  skill  development  must  be  an  integrated  part  of  an 
interdisciplinary  program  of  study  —  a  program  or  environment  that  places  students  in  situations  where  they 
can  practice  solving  problems  in  a  constructive  and  collaborative  manner. 

One  new  educational  strategy  that  addresses  the  above  problem  is  the  use  of  knowledge  construction 
environments  (KCEs).  One  kind  of  knowledge  construction  environment  is  the  computer-supported  intentional 
learning  environments  (CSILEs)  designed  to  support  intentional  learning  (Scardamalia,  Bereiter,  et  al.,  1989; 
Scardamalia  and  Bereiter,  1991).  Through  their  work  with  CSILEs  in  educational  settings,  Scardamalia  and 
Bereiter  found  that  through  the  development  and  support  of  a  collective  zone  of  proximal  development 
(cooperative  knowledge  building)  there  is  potential  for  students  to  move  toward  higher  levels  of  control  over 
their  zone  of  proximal  development.  In  other  words,  successful  student  participation  in  a  knowledge 
construction  environment,  such  as  a  CSILE,  can  help  students  become  more  responsible  for  the  process  of  learning, 
including  beconung  more  creative  and  flexible  problem  solvers. 

How  does  a  knowledge  construction  environment  work?  What  kinds  of  activities  do  students  engage  in?  The 
following  is  a  description  of  the  process  of  intentional  learning  promoted  in  a  CSILE  setting: 


Introduction 


111 


•  After  being  presented  with  a  content  area  for  investigation  (by  the  instructor  or  collaboratively  through 
group  process),  students  individually  determine  what  they  want  to  know  about  the  content  area  through 
the  development  of  low  level  (factual)  questions  and  high  level  (wonderment,  content  synthesis) 
questions. 

•  After  question  development,  students  construct  an  action  plan  specifying  what  their  goals  are  (e.g., 
answers  to  their  questions  developed  during  the  previous  stage)  and  a  plan,  including  a  timetable,  for 
gathering  the  necessary  information  and  completing  appropriate  projects  to  answer  their  questions. 

^>  After  gathering  information  and  completing  learning  acti\aties,  students  have  constructed  new  meaning 
(knowledge)  from  the  infonnation.  These  answers  are  placed,  using  text  and  graphics,  in  a  networked 
computer  program.  Then  the  students  review  each  other's  answers  to  see  if  the  questions  have  been 
answered  sufficiently.  As  a  result,  students  may  be  required  to  gather  more  information,  rework  their 
synthesis  of  information,  or  simply  rewrite/represent  (all  answers  are  not  necessarily  presented  in 
written  form  due  to  the  possible  use  of  graphics,,  digitized  voice,  video,  etc)  their  answers  for  clarity  or 
specificity. 

•  Finally,  students  must  fonnally  add  their  constructed  knowledge  to  a  collective  database  that  the 
students  are  building  together.  This  process  includes  developing  appropriate  links  to  other  information 
already  existing  in  the  database  so  that  other  students  may  be  able  to  access  the  new  information. 
Although  this  is  a  final  step,  students  are  responsible  for  maintaining  and  updating  the  database 
throughout  the  lifecycle  of  the  database. 

•  For  support  throughout  the  learning  process  students  engage  in  collaborative  conunenting  which  provides 
students  with  constructive  feedback  from  peers  (as  well  as  the  instructor)  during  question  development, 
goal  and  plan  construction,  information  gathering  and  s)aithesis,  and  answer  reporting  to  the  collective 
database. 

To  apply  Scardamalia  and  Bereiter's  study  on  CSILEs,  a  computer-supported  intentional  learning  environment 
was  used  to  teach  advanced  HyperTalk®  programming  techniques  (involving  high-level  reasoning  and 
problem-solving  skills)  and  CSILE  concepts  in  an  advanced  hypermedia/  multimedia  dass.  As  part  of  the  class 
requirements,  students  created  their  own  CSILE  programs  witltin  HyperCard®  (this  taught  the  HypeiTalk 
programming  techniques)  and  worked  collaboratively  within  their  CSILE  to  learn  knowledge  construction 
environment  concepts. 

The  use  of  a  CSILE  to  teach  about  knowledge  construction  enviromnents  came  about  as  a  reaction  to  student 
progress.  Because  of  the  complexity  and  richness  of  the  material,  students  were  having  trouble  understanding 
CSILE  concepts  through  the  coiwentional  process  of  learning  through  class  discussion  and  reading.  As  a  reaction 
to  this  problem,  it  was  proposed  that  an  understanding  of  CSILE  and  all  its  inherent  complexity  could  be  more 
easily  reached  if  students  actually  participated  in  a  CSILE. 

This,  of  course,  led  to  another  problem.  How  could  we  measure  the  effect  of  this  strategy?  We  were  interested  in 
seeing  whether  participation  in  a  knowledge  construction  environment  helped  students  develop  richer  and  more 
complex  knowledge  structures  about  CSILE.  To  examine  the  students'  knowledge  structures,  a  concept  map 
generating  program,  KNOT-Mac,  was  used  to  generate  representations  of  the  students'  maps.  Students 
generated  two  sets  of  concept  maps:  one  set  was  created  before  actual  CSILE  developnvent  began  and  a  second  set 
was  created  after  CSILE  development  and  in-class  participation  was  completed.  The  maps  were  then 
compared  to  each  other  as  well  as  to  an  expert's  concept  n>ap.  It  was  our  hypothesis  v/as  that  students  would 
develop  a  deeper,  more  expert-like  understanding  of  the  CSILE  concepts  by  seeing  how  a  CSILE  works 
firsthand.  In  other  words,  it  was  assumed  that  the  second  n\ap  generated  by  the  class  would  more  closely 
resemble  the  expert's  map  than  the  first  student  map  would. 
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Method 


Participants 

Participants  were  14  students  in  a  graduate  course  entitled  Hypermedia  and  Multimedia  Learning 
Environments.  Students  included  seven  nr^en  and  seven  women.  They  ranged  in  age  from  29  to  45.  About  half  were 
public  school  teachers  and  the  others  were  working  in  training  and  development.  Participation  in  this  study 
was  part  of  the  students'  class  participation  requirement. 

Concepts 

Seventeen  CSILE  concepts  (see  Table  1)  were  used  in  iixdividual  map  construction.  The  concepts  were  selected 
through  a  three  step  process: 

1.  Students  generated  a  list  of  concepts  that  they  believed  were  important  to  understanding  CSILE. 

2.  Two  experts  edited  the  student  concepts  and  added  some  of  their  own  creating  a  list  of  about  20 


3.    two  experts  practiced  making  comparisons  with  the  concepts.  From  that  practice  they  compiled  a  list 
of  17  concepts  for  the  concept  map  exercises. 


Table  1 

CSILE  Concepts  Used  in  Paired-<!omparison  Test 


It  should  be  noted  that  when  the  list  of  terms  key  to  the  overall  concept  of  CSILE  were  developed,  reference  to 
computers  or  other  media  were  purposely  excluded.  Computers  and  media  are  simply  tools  that  help  make  the 
environment  more  efficienit^  but  are  not  necessairy  elements  of  a  knowledge  construction  environment. 

Procedure 

Students  were  first  exposed  to  knowledge  construction  environments  and  CSILEs  in  particular  through  assigned 
readings.  Following  the  assigned  readings,  two  class  discussions  were  held  focusing  on  CSILEs  and  intentional 
learning.  After  the  class  discussions,  students  generated  their  first  concept  map. 

Students  generated  their  maps  during  class  time  using  the  Knowledge  Network  Organizing  Tool  for  the 
Macintosh  (KNOT-Mac).  KNOT- 

Mac  gatl\ers  data  through  a  paired-comparison  process.  The  program  presents  students  with  two  concept  terms 
at  a  time  in  which  they  rated  on  a  continuum  scale  from  "not  related"  to  "very  related"  (see  Figure  1).  Each 
concept  mapping  exercise  took  approximately  15  to  20  minutes  and  resulted  in  a  graphic  representation  that 
could  be  compared  to  other  students'  and  experts'  knowledge  structures  (see  Figure  2). 


concepts. 


active 
boring 

collaboration 

commenting 

control  of  learning 

goals/plans 

high  level  learning 

higher  order  questioning 


intentional  learning 
interesting 

knowledge  constniction 
low  level  learning 
lower  order  questioning 
passive 
student 

student  initiative 
teacher 
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Figure  1 

Rating  System  Used  to  Match  Concepts 


What  is  the  relationship  between: 

active 
intentional  learning 

I  I 

Low  High 


Students  generated  their  first  set  of  maps  after  reading  two  articles  about  CSILEs  (the  two  articles  referenced 
at  the  beginning  of  this  paper)  and  in-dass  discussions  of  those  articles.  The  second  set  of  maps,  done  at  the  end 
of •  _  'mester,  reflects  the  knowledge  structures  of  the  class  after  working  with  CSILEs  in  class  (specifically 
focusing  on  learning  environments  as  an  instructional  strategy). 

After  students  generated  both  sets  of  maps,  the  individual  student  maps  for  each  set  were  averaged  creating  one 
collective  map  for  each  set  (after  class  discussions  and  at  the  end  of  the  semester).  The  collective  maps  were 
used  for  data  results. 

Finally,  an  expert  map  was  created.  The  expert  map  was  created  through  a  collaborative  concept  rating  session 
with  two  knowledge  construction  environment  experts.  A  collaborative  strategy  was  used  because  there  were 
not  enough  experts  to  create  a  collective  or  averaged  map.  Both  experts  examined  each  pair  concept  terms 
presented  during  a  KNOT-Mac  session  identical  to  the  sessions  students  participated  in  and  discussed  the 
extent  of  the  relationship  between  the  concepts  and  then  selected  a  mutually  agreed  upon  value. 
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Resitits 


Results  of  the  concept  map  analysis  are  based  on  a  visual  comparison  and  interpretation  of  the  first  map  (Figure 
2)  with  the  second  (Figure  3)  and  each  session  with  the  expert  map  (Figure  4). 

Expert  Observations 


Figure  2 
Expert's  Map 
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1.  Intentional  learning  forms  the  most  central  concept  with  seven  direct  connections  (see  Table  2):  control 
of  learning,  collaboration,  teacher,  goals/plans,  high  level  learning,  higher  order  questioning,  and 
interesting.  These  are  the  main  attributes,  as  judged  by  experts,  related  to  intentional  learning  and 
knowledge  construction  environments. 

The  concept  "active"  provides  a  link  to  a  second  cluster  of  terms:  lower  order  questioning,  low  level 
learning,  boring,  and  passive.  This  indicates  ^ '^at  activity  is  the  basis  of  learning,  whether  high 
level  or  low  level.  However,  the  concept  "interesting"  is  not  linked  with  the  Iova*  level  cluster,  but 
with  the  intentional  learning  cluster.  "Interesting"  is  an  adjective  that  describes  ^nd  places  a  value  on 
high  level  learning. 

A  third  cluster  of  tenns  is  related  to  the  student.  "Commenting"  is  within  the  province  of  student 
activity.  "Control  of  learning"  provides  the  link  between  intentional  learning  and  the  student  because 
the  main  objective  of  intentional  learning  is  to  enable  students  to  take  more  control  over  their  learning. 


4. 


The  term  "knowledge  construction"  also  provides  a  set  of  connections  that  lead  to  the  construction  of 
knowledge  structures:  active,  higher  order  questioning,  and  high  level  learning.  "Knowledge 


construction''  is  also  at  the  top  of  the  map,  all  paths  lead  to  knowledge  construction,  the  goal  of 
intentional  learning. 


Overall,  the  experts'  map  shows  more  of  a  web-like  structure  than  either  of  the  class'  maps. 


Table  2 

Connections  in  the  Experts'  Map 

active 

3 

boring 

2 

collaboration 

1 

commenting 

1 

control  of  learning 

2 

goals/plans 

1 

high  level  learning 

2 

higher  order  questioning 

2 

intentional  learning 

7 

interesting 

2 

knowledge  construction 

3 

low  level  learning 

2 

lower  order  questioning 

2 

passive 

1 

student 

3 

student  initiative 

1 

teacher 

1 

Class  Observations:  First  Map 

In  general,  the  first  class  map,  generated  after  reading  two  articles  and  two  class  discussions,  does  not  show  any 
"mistakes,"  but  it  does  shows  a  lack  of  sophisticated  understanding  of  the  concepts  and  how  they  interrelate. 
This  unsophisticated  understanding  is  most  apparent  in  the  lack  of  any  central  focusing  concept,  such  as 
intentional  learning  in  the  experts'  map  and  in  the  linearity  of  most  of  the  relationships. 

1 .  Tlie  most  connected  concept  in  the  class'  first  map  is  higher  order  questioning  (see  Table  3).  Questioning 
was  one  of  the  most  discussed  concepts  in  the  Scardamalia  and  Bereiter  articles  so,  it  seems  logical 
that  they  would  focus  on  this  concept. 

2.  The  class  v/as  composed  of  mostly  teachers.  This  is  reflected  in  the  knowledge  construction"  - 
"teacher"  -  "student"  path.  This  infers  that  the  student  constructs  knowledge  through  the  teacher. 
This  indicates  a  lack  of  understanding  of  the  purpose  of  knowledge  cor\struction  enviromnents  and 
intentional  learning  in  turning  control  of  learning  over  to  the  student. 
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figure! 

first  Map  Produced  by  Class 
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3.  "Intentional  learning/'  the  most  central  concept  of  the  experts,  was  related  to  ''higher  order 
questioning"  and  "control  of  learning,"  but  did  not  form  the  basis  of  the  class'  first  niap. 

4.  The  concepts  of  "commenting"  and  "collaboration"  were  related  to  "active/'  but  there  was  no  direct 
connection  to  "intentional  learning."  This  indicates  tliat  these  concepts  were  understood  but  the 
relationship  to  intentior\al  learning  was  not  complete. 

All-in-all,  the  first  class  map  shows  an  understanding  of  the  concepts  as  individual  terms,  but  it  shows  a  lack 
of  depth  aiid  lack  of  understanding  about  how  the  concepts  interrelate.  For  example,  all  kinds  of  questioning 
lead  to  knowledge  construction,  but  in  the  class'  first  map  "knowledge  construction"  is  associated  only  with 
"higher  order  questioning." 
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Tables 

Connections  in  the  Class'  First  Map 


active 
boring 
collaboration 
commenting 
control  of  learning 
goals/ plans 
high  level  learning 
higher  order  questioning 
intentional  learning 
interesting 
knowledge  construction 
low  level  lean^ing 
lower  order  questioning 
passive 
student 
student  initiative 
teacher 


3 
2 


Class  Observations:  Second  Map 


Figure  4 

Second  Map  Produced  by  Class 
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1.  The  word  "acdve"  forms  a  central  focus  point  for  the  concept  relationships.  Active  has  seven  direct 
connections  (see  Table  4):  student  initiative,  intentional  learning,  collaboration,  goals/plans, 
interesting,  higher  order  questioning,  and  teacher.  The  concepts  connected  to  "active"  are  the  main 
attributes  of  learning  environments  and  intentional  learning.  The  word  "active"  is  central  because  this 
reflects  the  students'  point  of  views  and  experience.  They  were  active  in  questioning  and  collaborating 
and  developing  goals  and  plans.  Whereas  the  exp>erts  place  intentional  learning  as  the  central  concept 
because  it  is  the  basis  of  the  knowledge  construction  strategy. 


2.  The  word  "conunenting"  shows  a  change  in  conceptualization.  In  the  second  map  it  is  related  to 
"higher  order  questioning."  This  demonstrates  a  more  sophisticated  understanding  of  commenting, 
because  it  is  an  integral  part  of  the  questioning  strategy  in  a  CSILE. 

3.  The  cluster  related  to  "low  level  learning"  shows  a  slight  change  from  the  first  map  also.  In  the  first 
map,  the  class  simply  associated  those  concepts  together  in  a  long  chain.  In  the  second  map,  the 
students  refined  their  definitions,  associating  'lower  order  questioning"  with  "low  lovel  learning,"  but 
separating  "boring"  and  "passive."  This  indicates  a  greater  understanding  of  the  questioning  process, 
that  lower  order  questions  lead  to  low  level  learning  but  are  not  necessarily  boring  or  passive. 

4.  Another  difference  between  the  first  and  second  maps  is  the  position  of  "control  of  learning."  In  the 
second  map,  control  of  learning  is  related  to  goals/plans,  again  demonstrating  a  more  refined 
understanding  of  the  meaning  of  control  of  learning. 

5.  The  second  map  still  reflects  some  immaturity.  For  example,  "knowledge  construction"  is  related  to 
"student  initiative,"  but  to  nothing  else  that  leads  to  knowledge  construction,  such  as  higher  order 
questioning,  collaboration,  or  intentional  learning. 


In  sum,  the  class'  second  map  is  less  linear  than  the  first.  It  shows  that  after  working  with  CSlLEs,  the  class 
has  a  tetter  idea  of  the  interrelationships  among  the  concepts  and  a  deeper  understanding  of  the  topic  of 
intentional  learning.  They  have  formed  a  more  sophisticated  knowledge  structure  of  the  concept. 


Table  4 

Connections  in  the  Class'  Second  Map 
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intentional  learning 
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Conclusion 


Discovering  the  effectiveness  of  collaborative  learning  environments  using  recall  measures  is  a  fairly  basic 
task.  However,  examining  the  effect  of  those  environments  on  thought  processes  is  much  more  difficult  because 
it  involves  inferential  methods.  One  of  these  methods  is  the  use  of  maps  representing  the  perceived 
relationships  between  several  pairs  of  concepts  associated  with  a  specific  topic  using  pathfinder  or 
multidimensional  scaling  techniques.  This  study  exanriined  the  effect  of  working  within  a  computer  supported 
learning  environment  on  the  participants'  knowledge  structures.  The  findings  indicated  that  after  working 
with  the  learning  environment  the  participants'  knowledge  structures  became  more  complex  and  sophisticated. 

These  results  must  be  utilized  carefully,  for  the  basic  findings  are  dependent  upon  observation  and  visual 
analysis  instead  of  probability  levels.  Reasons  for  the  changes  in  the  knowledge  structure  are  speculations 
based  on  observations  rather  than  on  testing  data.  However,  the  differences  among  the  three  maps  compared 
are  striking.  The  next  step,  then,  is  to  utilize  additional  measures  to  see  if  the  complex  visual  representation  of 
the  knowledge  structure  can  be  associated  with  performance  in  real-world  tasks.  For  example:  does  a  complex 
knowledge  structure  mean  that  a  person  may  be  able  to  use  the  knowledge  represented  in  a  more  flexible  and 
sophisticated  way?  Further  information  is  also  needed  to  explain  the  difference  between  the  expert's  map  and 
the  complex  learner's  map.  Why  is  it  that  the  learners  had  a  different  focus  point  in  their  map?  The  use  of  the 
maps  to  identify  learner  progress  in  creating  knowledge  structures  seems  to  provide  a  promising  new  way  of 
evaluating  the  effectiveness  of  instrucUotval  techniques. 
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Using  Multimedia  to  Provide  Information  About  Childhood  Cancer  to  Elementary  Grades 

Ida  Flymi,  University^  of  Pittsburgh 

This  showcase  will  allow  users  to  access  information  about  childhood  cancer  from  a  multimedia 
interactive  information  retrieval  system  designed  and  developed  for  elementary  grade  students. 

This  videodisc  contains  30  minutes  of  selected  segments  from  the  special  taped  developed  by 
Family  Communications  and  the  American  Cancer  Society;  its  format  is  visual  and  audio 
intensive.  The  computer's  hard-disk  contains  a  factual  database  related  to  specific  childhood 
cancers  and  their  treatment  information  on  support  groups  and  a  list  of  children's  books  available 
in  local  public  libraries.  The  interface  between  the  two  databases  gives  users  control  in 
retrieving  discrete  chunks  of  information,  from  either  database,  based  on  their  current  needs. 
Specific  search  paths  can  take  users  from  one  database  to  the  other  via  existing  links  created  in 
the  interface  and  activated  by  mouse  clicks.  Users  may  look  at  the  same  segment,  from  either 
storage  medium,  as  often  as  needed.  They  may  jump  back  to  a  previous  segment  or  forward 
to  a  new  one  within  a  matter  of  seconds. 

It  is  hoped  that  viewers  will  be  interested  in  evaluating  the  retrieval  system  and  volunteer  their 
ideas  and  suggestions  on  its  design  and  contents. 

Computer-Assisted  Learning  -  Tools  vSo  Appliances 

John  A.  Mack,  XR  Data  Systems 

Since  the  initial  excitement  generated  in  the  1960's  by  the  potential  for  computer  u^ge  in 
education,  virtually  all  computer-assisted  learning  materials  have  been  produced  by  content 
experts  for  use  by  individual  learners.  Although  the  person-hour  cost  of  an  hour  of  courseware 
had  remained  remarkably  constant,  computer  technology  has  changed  dramatically.  Thousands 
of  computers  have  become  millions,  and  they  are  nearly  as  accessible  to  end-users  as  paper  and 
pencil. 

Although  hypertext,  personal  information,  or  text  base  management  programs  may  be  useful  for 
organizing  knowledge,  it  does  not  follow  that  they  would  be  particularly  useful  as  tools  for 
learning;  an  important  characteristic  of  educational  software  is  that  it  is  intended  to  put 
something  into  the  memory  of  the  learner  (not  just  into  the  memory  of  the  computer).  The  high 
person-hour  cost  of  courseware  suggests  that,  in  order  to  be  useful,  authoring  tools  for  learners 
are  likely  to  be  quite  different  from  authoring  tools  for  teachers. 

A  package  of  programs  (X-CAR)  will  be  described  which  supports  computer  usage  by  an 
individual  learner  with  an  arbitrary,  self-defined,  and  evolving  knowledge  base  to: 
-manage  that  knowledge  base, 

-produce  courseware  appropriate  to  that  knowledge  base,  and 
-manage  the  use  of  that  courseware  to  achieve  self-established  goals  of 
proficiency. 

123 


123 


Student  Access  to  Hospital  Information  Systems:  Problems  and  Approaches 

Cynthia  M.  Mascara  and  Ramona  Nelson,  University  of  Pittsburgh  Medical  Center 


Students  in  health  care  disciplines  learn  to  care  for  patients  through  contact  with  actual  clients. 
Like  all  health  professionals,  they  use  the  patient's  medical  record.  The  introduction  of 
computerized  patient  records  is  limiting  their  access  to  vital  patient  information. 

The  University  of  Pittsburgh  Medical  Center  and  the  University  of  Pittsburgh  School  of  Nursing 
piloted  a  project  providing  fifty-five  nursing  students  with  access  to  the  hospital  information 
system  (HIS).  The  major  issues  and  potential  approaches  to  providing  student  access  were 
grouped  into  five  areas.  These  include: 

1)  Level  of  Access  -  Students  may  be  given  the  same  access  as  professionals  or  a  more 

restricted  access. 

2)  Student  Instniction  -  Instruction  may  be  provided  by  hospital  employees  or  school 

instructors. 

3)  Teaching  Approach  -  Lecture,  demonstration,  hands-on  experience,  self-learning 

techniques  or  a  combination  may  be  used. 

4)  Data  Security  -  Generic  student  access  or  personal  access  codes  may  be  used. 

5)  EvaV^^tion  -  Techniques  for  evaluating  student  learning  and  benefits  of  access  must 

developed. 

This  project  demonstrated  that  cooperative  planning  and  careful  evaluation  of  issues  and  potential 
solutions  is  imperative  when  providing  short-term  student  access  to  automated  patient  data.  This 
pilot  has  been  instrumental  in  planning  HIS  access  as  a  routine  component  of  health  care  at 
UPMC. 


Clinical  Nursing  Experience  in  Labor  and  Delivery  Via  Interactive  Video 

Dee  McGonigle,  Pennsylvania  State  University 

Interactive  video  can  augment,  enhance,  and  even  replace  clinical  nursing  education  depending 
upon  the  learner's  needs.  The  clinical  area  of  labor  and  delivery  is  an  educational  challenge; 
providing  students  with  clinical  experience  is  difficult.  The  time  spent  in  labor  and  delivery  is 
limited  and  the  variety  of  births  obsei-ved  is  not  predictable.  The  presenters,  are  nursing 
researchers  who  obtained  grant  monies  and  developed  an  independent  learning  module  in  which 
all  of  the  nursing  students  receive  the  same  labor  and  delivery  clinical  exp>eriences  via  interactive 
video.  This  learning  modality  is  applicable  to  generic  students,  returning  registered  nurses 
seeking  baccalaureate  degrees,  graduate  nursing  students,  and  nurses  currently  working  or 
seeking  positions  in  this  clinical  setting.  Utilizing  this  technological  learning  modality  will 
enhance  the  students'  experiences  and  ultimately  improve  professional  practice  and  client  care. 
This  presentation  will  review  the  grant  guidelines  and  proposal.  The  pilot  program  design  will 
be  explored  with  a  focus  on  the  targeted  research  areas.  A  video  presentation  of  the  pilot 
program  will  be  assessed  and  the  evaluation  procedures  reviewed. 
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Faculty-Student    Teams    Develop    CBL  Software 

Anthony  J.  Frisby,  Ph.D. 
Rodney  B.  Murray,  Ph.D. 
Thomas  Jefferson  University 

Abstract 

The  Summer  Computer  Fellowship  Program  (SCFP)  was  established  by 
Jefferson  Medical  College  in  1984.    The  primary  objective  of  the  fellowship 
is  to  provide  an  opportunity  for  faculty  and  students  to  learn  about  the  use 
of  computers  in  medical  education  and  to  produce  computer  programs  that 
could  be  used  by  future  students  as  part  of  the  curriculum.  Competition 
for  the  fellowship  program  is  open  to  any  medical  student  upon  completion 
of  their  freshman  year. 

Faculty  preceptors  who  have  submitted  proposals  for  educational 
computer  projects  (also  accepted  on  a  competitive  basis)  are  then  paired 
with  a  fellow.    The  teams  spend  the  next  ten  weeks  working  on  computer 
based  learning  (CBL)  programs.    The  Office  of  Academic  Computing  (OAC) 
provides  guidance  in  instructional  design  and  educational  principles,  as 
well  as  programming  assistance. 

The  development  of  effective,  attractive,  and  high  quality  instructional 
programming,  is  a  time  consuming  and  difficult  undertaking.    The  SCFP 
students  work  hard  to  develop  instructional  programs  that  meet  the  needs 
of  Jefferson's  medical  students.    During  the  past  eight  years  the  SCFP  has 
produced  a  wide  variety  of  programs,  many  of  which  have  become  a 
significant  part  of  the  curriculum.    The  SCFP  continues  to  provide  CBL  for 
medical  and  allied  health  students. 

Summer    Computer    Fellowship  Program 

The  SCFP  was  established  at  Jefferson  Medical  College  in  1984  to  give 
faculty  and  students  an  opportunity  to  learn  about  the  use  of  computers  in 
medical  education  and  to  produce  computer  programs  that  could  be  used 
by  future  students  as  part  of  the  medical  curriculum.     Medical  students 
who  have  completed  their  first  year  are  eligible  to  apply  for  the  fellowship 
and  are  selected  on  a  competitive  basis. 
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Faculty  members  interested  in  participating  and  developing  a  CBL  project 
submit  proposals  to  the  OAC  which  reviews  and  selects  them  based  on 
several  criteria.    Among  those  criteria  are:  feasibility  (is  this  a  project  the 
fellowship  can  accomplish);  usefulness  (are  the  end  users  likely  to  use  the 
program);  and  need  (has  a  program  already  been  produced  that  would 
likely  fit  this  need).    Fellows  are  then  paired  with  faculty  preceptors  and 
the  resulting  teams  work  together  for  ten  weeks. 

In  the  past,  the  students  have  provided  much  of  the  computer  expertise 
and  performed  the  bulk  of  the  design  and  programming  of  the  CBL 
projects.    Recently  though,  more  of  the  faculty  preceptors  have  been 
returning  to  the  SCFP  to  produce  additional  programs  or  to  enhance  a 
previous  version. 

With  each  successive  year,  the  amount  of  personal  experience  in 
computing  has  increased  among  the  participating  faculty  members.  This 
increase,  along  with  the  significant  improvements  in  authoring  tools  and 
peripheral  devices  have  permitted  the  development  of  several  marketable 
programs. 

Since  1984,  thirty-three  faculty  and  sixty  students  have  participated  in  the 
SCFP.    Forty-six  different  projects  have  been  worked  on  of  which  19  are  in 
use  at  this  time.    Twelve  (36%)  of  the  faculty  have  participated  more  than 
once.    A  listing  of  programs  or  research  areas  appears  in  Appendix  A. 

There  "are  a  number  of  factors  that  affect  the  current  use  of  those 
programs  developed.    First,  not  all  the  teams  had  the  goal  of  developing  a 
software  application.    In  other  instances,  the  program  was  never  finished. 
Failure  to  complete  a  program  was  usually  because  the  student  ran  out  of 
time  and  couldn't  finish  it  during  the  school  year,  or  the  faculty  member 
no  longer  was  interested  in  it  (or  moved  to  another  institution). 

Process 

The  SCFP  runs  for  approximately  10  weeks,  meeting  as  a  group  regularly. 
In  that  time  we  introduce  the  students  to  the  fundamentals  of  instructional 
design,  provide  an  orientation  to  computers  (if  necessary),  and  teach  an 
authoring  language,  programming  language,  presentation  or  research  tool. 
With  these  skills  the  students  and  preceptors  work  on  their  projects,  the 
OAC  providing  technical  assistance  and  equipment  resources.  Periodically 
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during  the  10  week  program  students  will  demonstrate  their  programs  to 
the  whole  group  to  critique  and  share  ideas. 

Realizing  that  10  weeks  is  a  very  short  time  to  develop  a  viable  piece  of 
CBL,  we've  structured  the  program  to  optimize  the  time  available.  The 
first  two  weeks  are  formally  structured,  providing  lectures  and  materials 
on  instructional  design  and  tutoring  on  an  authoring  (or  other)  language. 
Students  are  expected  to  spend  at  least  eight  hours  a  day  working  on  their 
projects.    Starting  with  the  third  week,  we  meet  twice  a  week  to  discuss 
any  problems,  concerns,  and  new  insights. 

Conclusion 

This  year,  1992,  marked  the  eighth  anniversary  of  the  computer 
fellowship  program.    After  each  summer,  a  short  evaluation  has  been 
conducted  to  assess  the  overall  feeling  regarding  the  SCFP.    Both  faculty 
and  students  strongly  agree  that  the  SCFP  is  a  useful  and  enjoyable 
experience.    They  agreed  that  the  program  helped  increase  their 
knowledge  of  computers  and  their  use  in  medicine.    Most  agreed  that  they 
had  produced  a  useful  program. 

The  support  and  positive  feedback  have  encouraged  us  to  pursue 
continued  funding  for  the  program.    Jefferson  remains  committed  to  the 
computer  fellowship  program  and  the  important  role  computers  play  in 
education,  practice  and  research. 


Appendix  A 


Summer  Fellowship  Program  Projects 


Q&A  Respiratory  Histology 
Tutorial  on  Abnormal  Karotypes 
Disease  Model  for  Patient  Simulator 
Database  of  Review  Questions  in  Biochemistry 
Self-Study  Guide  for  Computer  Novices 
Translate  Case  Study  (basic  to  pilot) 
Tutorial/Simulation  on  Pediatric  Poisoning 

laM 

Multiple  Choice  Anatomy  Review 
Diagnostic  Database  for  Expert  System 
Patient  Simulation  Authoring  System 
Videodisc  Control  Programs 
Review  Questions  on  Carbohydrate  Metabolism 
Health  Screening  Tutorial/Simulation 

Tutorial  on  Sensitivity,  Selectivity  and  Predictive  Value  of  Lab  Tests 
1§M 

Muscle  Attachment  Tutorial 
Tutorial  -  Adult  Immunization 
Tutorial  -  Respirator  Mechanics 
Continuation  of  Tutorial  on  Lab  Tests 
Tutorial  -  Facial  Dysmorphism 
Natural  Language  Q&A  in  Gross  Anatomy 
Tutorial  -  Computerized  Medical  Record 
Authoring  Program  for  Videodisc  Programs 

1997 

Case  Simulation  for  Emergency  Medicine 
Patient  Management  Problem:  Alcoholism 
Explore  use  of  Video  in  Teaching  Microbiology 
Tutorial  &  Cases  on  Neurology 
Authoring  Program  for  Videodisc  Quiz 
Automated  Pediatric  Physical  Exam  Entry 
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Tutorials:  Interview  Substance  Abuser 
Histology  Tutorials 
Clinical  simulations  OB/GYN 
Spinal  Injury 

Promoting  Health  in  Medical  Practice 
Tutorial/Reference  in  Psychological  Development 
Electrical  properties  of  cell  membranes 


12M 

Diagnosing  Substance  Abuse 

Pathology  of  Inflammation 

CAl-V  Histology  Tutorial  and  Quiz 

Geriatric  Curriculum  Project 

The  Gross  Project 

Cardiac  Pathology 

Child  &  Adolescent  Behavior  Eval. 

1990 

Antibiotics  Guide  (3  modules,  3  students  &  3  faculty) 
The  Gross  Project 
Histology  Tutorials 

1991 

Histology  Quiz  Maker 

Diagnostic  Problems  in  Child  Development 

Pathology  Quiz  Bank 

The  Gross  Project 

Diabetic  Patient  Education 

Skeletal  Muscle  Contraction 

MECCA 

Microbiology  Guide  Notes 
1992 

The  Gross  Project/A. D. A.M. 
Simulations  in  Microbiology 
Q&A  Pathology 
Q&A  Histology 
Tutorial  -  pH 

Intudcution  to  Ophthalmology 
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GUroELINES  FOR  DESIGNING  EFFECTIVE  AND  HEALTHY  LEARNING 
ENVIRONIVIENTS  FOR  INTERACTIVE  TECHNOLOGIES 


Michael  Weisberg,  National  Library  of  Medicine 


ABSIRACT 


Many  of  the  findings  from  ergonomics  research  on  visual  display  workstations  are  relevant  to 
the  design  of  interactive  learning  stations.  This  paper  briefly  reviews  ergonomics  research  on 
visual  display  workstations;  specifically,  (1)  potential  health  hazards  from  electromagnetic 
radiation;  (2)  musculoskeletal  disorders;  (3)  vision  complaints;  and  (4)  psychosocial  stresses. 
Based  on  this  review,  guidelines  on  how  to  design  an  ergonomically  correct  workstation  and 
learning  environment  are  presented.  These  guidelines  seek  to  balance  human  performance 
with  learner  satisfaction  and  well-being. 
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mTRODUCTION 


Technological  advances  in  recent  years  have  led  to  the  prevalence  of  computer  and  other 
electronic  technologies  in  the  everyday  lives  of  people-at  play,  at  work,  in  the  home  and  in 
school  (Forester,  1989;  Fox,  1989;  Postman,  1352).  Recent  technology  innovations  include 
the  "Personal  Digital  Assistant"  (Linderholm,  1992)  and  the  inchoate  fields  of  cyberspace 
(Gibson,  1984;  Benedikt,  1992)  and  virtual  reality  (Rhiengold,  1991).  As  technologies 
insinuate  themselves  into  work,  entertainment  and  learning  environments,  the  interests  of 
broadcasting,  publishing,  and  computer  industries  intersect  and  become  the  focal  point  for  a 
variety  of  interactive  systems  (Brand,  1988).  These  systems  may  include  visual  display 
monitors,  microcomputers,  laser  discs,  various  optical  disk  formats  and  a  plethora  of 
peripherals  (Frenkel,  1989).  As  these  technologies  become  more  widely  used  certain 
problems  are  emerging.  These  problems  range  from  the  slightly  humorous  (the  images  of 
frustrated  consumers  attempting  to  program  a  video  cassette  player  to  record  their  favorite 
television  program)  to  the  very  serious,  e.g.,  (potential  health  hazards  from  electromagnetic 
fields  emitted  by  visual  display  devices,  and  the  physical  and  psychological  stress  of  spending 
long  hours  working  or  studying  at  computer  workstations). 

Much  research  addressing  the  potential  and  real  problems  associated  with  the  design  and  use 
of  visual  display  workstations  has  been  conducted.  The  majority  of  research  has  focused  on 
the  productivity  and  health  of  workers  who  use  visual  display  workstations:  (NIOSH,  1991; 
Sauter,  1991;  EPA,  1990;  Klemmmer,  1989;  Kilbom,  1988;  Zenz,  1975).  Numerous 
guidelines  for  the  design  of  effective  and  safe  workstations  have  emerged  from  this  research: 
IEEE,  Spectrum:  Special  Workstations  Guide  (1991);  NIOSH  Publication  on  Video  Display 
Terminals  (1991);  Computer  Workplace  Ergonomics  (Gross  and  Hassel  1991);  Herman 
Miller  Research  and  Design  (1991);  American  National  Standard  for  Human  Factors 
Engineering  of  Visual  Terminal  Workstations  (1988);  Boeing  Aerospace  Company  (1984); 
Human  Factors  of  Workstations  with  Visual  Displays,  (IBM,  1984);  and  Design  of  Visual 
Display  Products-Workstation  Ergonomics,  (IBM,  1988).  A  somewhat  lesser,  although 
substantial,  amount  has  been  written  about  the  real  or  potential  problems  associated  with  the 
design  and  use  of  technology  in  the  home  (Miles,  1988;  Norman,  1988;  Bass,  1991).  Very 
little,  however,  has  been  written  about  problems,  either  potential  or  real,  associated  with 
using  visual  display  technologies  designed  for  use  in  learning  environments  (Spangenberg, 
1975;  feaman,  1983,  1984,  1986). 

Learning  environments,  of  course,  differ  from  the  workplace  environment.  In  the  learnplace 
environment,  the  measure  of  productivity  is  not  a  unit  of  work  but  rather  the  accomplishment 
of  an  educational  objective,  usually  demonstrable  in  knowledge  or  performance.  The  design 
and  implementation  of  the  learning  environment  is  an  important  factor  in  achieving 
educational  objectives.  Interactive  technologies  are  an  important  segment  of  the  learning 
environment;  they  include  the  full  range  of  optical  formats  (laser  videodiscs,  CD-ROM,  CD- 
ROM  XA,  CD-I,  CD-TV,  and  DVI)  and  microcomputers  and  peripherals  that  are  currently 
available.  Also  included  are  those  technologies  that  are  on  the  horizon  and  those  that  can  be 
reasonably  anticipated  for  the  near  future,  such  as  multimedia  systems. 
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The  learnplace  environment  for  interactive  technologies  has  many  similarities  with  the 
workplace  environment  for  visual  display  workstations.  Among  these  similarities  are  such 
dynamics  as:  (1)  viewing  a  visual  display  monitor;  (2)  using  a  mechanical  device  (keyboard, 
mouse,  trackball )  or  a  touch  screen  monitor;  (3)  sitting  for  long  periods  in  awkward 
positions  while  interacting  with  devices  and  furniture  configurations;  and  (4)  performing 
under  stress.  Many  of  the  findings  from  ergonomics  research  on  visual  display  workstations 
are  relevant  to  the  design  of  interactive  learning  stations. 

The  paradigm  and  hypotheses  guiding  research  on  the  effective  design  and  use  of  visual 
display  workstation  technologies  were  derived  from  the  principles  of  ergonomics. 
Ergonomics  is  an  interdisciplinary  science  that  addresses  human  performance  and  well-being 
in  relation  to  the  job  (task),  the  equipment,  and  the  environment  (Gross  and  Hassel,  1991). 
Ergonomics  embraces  three  major  concerns:  (1)  safety  and  health;  (2)  comfort  and  well- 
being;  and  (3)  productivity  and  efficiency  (Bems  and  Claridge  1990).  The  scientific 
disciplines  involved  in  ergonomics  fall  under  three  general  headings:  (a)  engineering  and 
physical  sciences;  (b)  biological  sciences;  and  (c)  behavioral  sciences,  including  psychology 
(Zenz,  1975).  These  disciplines  define  the  domain  for  research  on  ergonomics. 


A  useful  conceptual  framework  for  opcrationalizing  this  domain  is  found  in  Human  Factors, 
Ergonomics  and  Usability,  (Baker,  1989).  Baker  describes  a  "model  of  work"  proposed  by 
IBM  (1982)  and  Baker  (1984)  that  explores  the  work  environment  by  delineating  the 
relationships  among:  (a)  the  individual  user;  (b)  the  tools;  (c)  the  workstation;  and  (d)  the  job 
to  be  performed.  With  the  substitution  of  "learner"  for  "user"  and  "learning/performance" 
for  "job  to  be  performed",  this  model  is  easily  adaptable  to  the  learnplace.  Figure  1. 
illustrates  the  model  with  the  proposed  substitutions. 
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The  model  addresses  human  performance,  satisfaction  and  well-being  in  relation  to  the  task, 
the  technology  and  the  environment.  It  suggests  interrelationships  among  the  major  variables 
and  provides  an  ergonomic  framework  for  defining  and  investigating  the  extent  of  these 
relationships. 

Much  of  the  recent  research  on  the  ergonomic  aspects  of  work  with  visual  display 
workstations  explores  the  relationships  among  the  major  variables  as  suggested  by  the  IBM 
(1982)  and  Baker  (1984)  model.  The  lion's  share  of  the  research  examines  the  potential 
health  hazards  to  users  from  exposure  to  electromagnetic  radiation  emitted  by  visual  display 
terminals.  Since  1977,  NIOSH  alone  has  conducted  thirty-six  health  haz^jd  evaluations  and 
published  seventy-two  scientific  articles  (Sauter,  1990).  Other  studies  examine  the  effects  of 
video  display  terminal  work  on  the  musculoskeletal  and  vision  systems,  and  on  the  general 
stress  level  of  workers  (Amdt,  1984). 

The  remainder  of  this  paper  will  review  briefly  the  recent  ergonomics  research  on  visual 
display  workstations.  Research  areas  will  include:  (1)  potential  health  hazards  from 
electromagnetic  radiation;  (2)  musculoskeletal  disorders;  (3)  vision  complaints;  and  (4) 
psychosocial  stresses.  Based  on  this  review,  guidelines  on  how  to  design  an  ergonomically 
correct  workstation  and  learning  environment  will  be  suggested  with  the  goal  of  preventing 
potential  health  problems  and  optimizing  human  performance.  These  guidelines  will  seek  to 
balance  human  performance  with  learner  satisfaction  and  well-being.  The  guidelines  will 
enable  health  professionals  to  design  learning  workstations  free  from  such  computer 
workplace  problems  as  musculoskeletal  disorders,  blurred  vision,  fatigue,  headaches,  and  the 
potentially  dangerous,  extremely  low  frequency  magnetic  fields  (ELF). 

VISUAL  DISPLAY  TERMINAL  HAZARDS 

A  dichotomy  of  opinion  exists  today  on  the  subject  of  potential  health  hazards  from  video 
display  terminals  (VDTs).  One  set  of  opinions,  represented  by  the  public,  is  informed  by 
popular  literature.  The  other,  represented  by  the  scientific  community,  is  informed  by 
epidemiological  studies.  This  makes  for  an  interesting  dilemma— the  public  is  worried 
although  the  preponderance  of  scientific  evidence  indicates  no  conclusive  evidence  of  harmful 
health  effecls  from  VDTs. 

Public  concern  over  the  potential  health  hazards  of  electromagnetic  radiation  emitted  by 
visual  display  terminals  was  aroused  by  the  publication  of  several  articles  in  popular 
magazines  and  newspapers:  (Brodeur,  1990;  Borrell,  1990;  Infoworld,  1990;  San  Jose 
Mercury,  1990;  Washington  Post,  1990).  Some  of  these  articles  had  very  provocative  titles: 
Is  Your  Computer  Killing  You,  (Borrell)  and  The  Magnetic  Field  Menace.  (Brodeur).  These 
articles  drew  attention  to  possible  health  hazards  from  very  low-frequency  (VLF)  and 
extremely  low-frequency  (ELF)  electromagnetic  fields  emitted  by  visual  display  terminals. 
Magnetic  fields,  or  more  specifically,  magnetic  flux  densities  historically  have  been  measured 
with  a  unit  called  the  milligauss  (1  milligauss(mg)=  .OOlGauss(g)  (Feero,  1991).  Electrical 
engineers  and  physicists  use  the  Tesla  as  a  unit  of  international  standard  (Feero,  1991).  One 
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Tesla  is  equivalent  to  10,000  Gauss  or  10,000,000  milligauss.  The  Tesla  is  used  in  technical 
journals  and  the  milligauss  unit  is  used  in  information  for  the  general  public. 

Public  articles  also  focused  on  the  concern  of  European  countries,  particularly  Sweden,  with 
the  development  of  guidelines  for  radiation  emissions  from  VDTs.  It  was  noted  that  in  1986 
Sweden  developed  guidelines  for  VLF  magnetic  emissions  (defined  as  frequencies  between 
2kHz  and  400kHz,  in  cycles  per  second,  and  that  these  had  been  amended  in  1990  to  include 
guidelines  for  ELF  magnetic  fields  (  defined  as  5Hz  to  2kHz  to  an  intensity  of  no  more  than 
2.5mg  at  one-half  a  meter  from  the  monitor)(0'Connor,  1991).  This  vigilance  in  Sweden 
spurred  concern  in  the  United  States. 

Scientific  research  indicates  that  video  display  terminals  cause  four  types  of  electrical  and 
magnetic  fields:  (1)  an  electrostatic  field  associated  with  high  voltage  applied  to  the  internal 
surface  of  the  cathode  screen;  (2)  a  magnetic  field  of  extremely  low-  frequency  generated  by 
the  vertical  scanning  coil;  (3)  a  low-  frequency  magnetic  field  generated  by  the  horizontal 
scanning  coil;  and  (4)  a  low-frequency  electrical  field  generated  by  the  high  voltage  flyback 
transformer  (Laliberte,  1987;  Maiha,  1986).  This  results  in  VLF  and  ELF  emissions  from 
the  back  and  sides  of  monitors.  These  emissions,  therefore,  have  been  the  focal  point  of 
concern  and  research. 

The  National  Institute  for  Occupational  Safety  and  Health  (Murray,  1991)  reports  three 
specific  health  problems  that  are  generally  attributed  to  operators'  exposure  to  radiation 
emitted  from  VDTs:  (1)  cataracts;  (2)  reproductive  problems;  and  (3)  facial  skin  rashes. 
Both  animal  and  human  studies  demonstrate  that  the  aforementioned  health  problems  can 
result  from  exposure  to  a  high  level  of  radiation  similai*  to  the  type  emitted  by  VDTs 
(NIOSH  1991).  However,  dozens  of  radiation  surveys  conducted  by  NIOSH  and  independent 
groups  such  as  Bell  Laboratories,  Duke  University  School  of  Medicine  and  the  University  of 
Washington  have  not  shov/n  any  scientific  evidence  that  the  occurrence  of  cataracts,  birth 
defects,  miscarriages,  or  skin  rashes  is  related  to  radiation  exposure  from  VDTs  (NIOSH, 
1991).  In  the  final  report  of  An  Investigation  of  Electrical  and  Magnetic  Field  and  Operator 
Exposure  Produced  by  VDTs:  NIOSH  VDT  Epidemiology  Study  by  Robert  Tell  and 
Associates,  it  was  rqx)rted  that  typical  personnel  exposures  to  electric  and  magnetic  fields 
are:  (1)  relatively  low;  (2)  within  a  relatively  confined  range  of  magnitudes;  (3)  not  highly 
dissimilar  to  exposures  commonly  encountered  from  radios  and  other  devices  routinely  found 
in  the  home  or  workplace;  and  (4)  generally  substantially  less  than  any  electromagnetic  field 
exposure  limits  developed  for  radiation  protection  purposes  by  organizations  within  the 
United  States  and  many  other  countries. 

NIOSH  (Murray,  1991)  also  addressed  the  issues  of  whether  malfunctioning  VDTs,  aging 
VDTs  and  the  presence  of  multiple  terminals  cause  increased  radiation  and  therefore  poses 
more  danger.  It  was  found  that:  (1)  malfunctions  would  make  a  VDT  unusable;  (2)  exposure 
does  not  seem  to  increase  as  terminals  become  older;  and  (3)  multiple  terminals  do  not 
increase  exposure. 
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Charles  Reis  (1991)  tried  to  put  the  issue  in  perspective  in  an  article  entitled,  Video  Display 
Terminal  Hazards:  The  Other  Side  of  the  Controversy,  His  article  was  a  balanced  attempt  to 
review  both  popular  press  articles  and  epidemiological  studies  conducted  by  industry, 
academia,  and  government  agencies.  Reis  concludes  that  it  is  an  almost  impossible  task  to 
draw  a  sound  conclusion  on  the  VDT  hazard  issue  and  suggests  more  research  on  the  long 
term  effects  of  exposure  to  VDTs.  This  is  both  sensible  and  prudent.  Further  research  is 
necessary  to  determine  the  long  term  effects  on  users  from  exposure  to  visual  display 
monitors.  Long  term  studies  make  sense  since  children  and  people  of  all  ages  are  being 
introduced  to  technology  and  are  certain  to  be  exposed  to  visual  display  monitors  over  the 
long  term.  This  is  not  to  say  that  liquid  crystal  displays  and  other  display  technologies,  which 
emit  no  radiation,  could  not  at  some  future  time  replace  standard  cathode-ray  tube  type 
monitors  and  end  the  dilemma. 

The  scientific  community  also  supports  research  on  the  long  term  effects  of  exposure  to 
VDTs.  NIOSH  supports  long  term  research  predicated  on  the  need  to  accurately  measure  the 
levels  of  radiation  exposure  for  users  and  to  compare  these  levels  to  existing  occupational 
exposure  standards  and  to  the  threshold  for  biological  effects  available  from  epidemiological 
studies  (Murray,  1991). 

The  question  becomes,  what  should  users  do  until  there  is  definitive  scientific  evidence  that 
visual  display  terminals  do  not  pose  a  long  term  health  hazard.  Preventive  measures  offer  a 
logical  solution.  The  Swedish  guidelines,  which  are  also  recommended  by  the  Institute  of 
Electrical  and  Electronics  Engineers,  Inc.,  are  a  good  general  reference  point.  Recent  studies 
of  some  commercial  monitors  found  that  field  strength  diminishes  with  distance  and  that  it  is 
generally  below  1  milligauss  at  76  cm  from  the  monitors  front  and  122  cm  from  its  sides  and 
back  (Spectrum,  1991).  This  finding  is  consistent  with  information  reported  in  the  Brodour 
(1990)  article  appearing  in  MACWORLD.  Taken  together,  these  findings  suggest  that  the 
user  is  exposed  to  ELF  levels  of  less  than  1  milligauss  when  sitting  29  inches  from  the  front 
of  the  display  and  4  feet  from  the  sides  or  back  of  the  display.  And  since  an  office  partition 
will  not  shield  users  from  ELF  emissions,  workspaces  should  be  designed  so  that  no  one  in 
an  adjoining  area  is  within  4  feet  of  the  sides  or  back  of  a  monitor  in  someone  else's  work 
area  (Spectrum,  1991). 

It  should  be  noted  that  there  have  been  some  false  claims  that  anti -glare  screens  reduce  low 
level  magnetic  fields.  When  properly  grounded,  anti-glare  screens  reduce  electrical  emissions 
but  they  do  not  significantly  reduce  low-level  magnetic  fields  (Branscum,  1990).  Magnetic 
field  elimination  or  reduction  through  shielding  requires  use  of  high  permeability  materials 
(Laliberte,  1987).  According  to  Laliberte,  shielding  against  magnetic  fields  has  been  obtained 
by  adding  a  semi-transparent,  high  permeability  filter  to  the  screen.  Some  monitor 
manufacturers  have  resorted  to  various  shielding  or  cancellation  techniques  for  reducing 
magnetic  emissions.  Although  these  efforts  are  represented  by  only  a  few  large  companies 
(IBM  and  SuperMac),  an  after  market  appears  to  be  developing  (O'Conner,  1991)  and  a 
recent  advertisement  in  Monitor  Radiation  News  (Vol.  1.,  1991).  This  development  has  the 
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ironic  effect  of  perpetuating  concern  about  radiation  while  simultaneously  providing 
preventive  measures. 

MUSCULOSKELETAL  DISORDERS  IN  A  VISUAL  DISPLAY  ENVIRONMENT 

Musculoskeletal  disorders  represent  a  complex  phenomenon.  Within  the  context  of  computer 
work  environments,  the  individual  disorders  themselves  are  complex  and  represent  a  wide 
array  of  conditions  that  are  commonly  referred  to  as  cumulative  trauma  disorders  (CTDs)  or 
repetitive  strain  injuries  (RSIs).  Research  shows  that  computer  work  and  the  design  of  the 
workstation  can  interact  to  amplify  the  risks  of  musculoskeletal  disorders  such  as  carpal 
tunnel  syndrome  and  other  cumulative  trauma  disorders.  Statistics  on  organizations  where 
computers  and  keyboards  are  essential  for  productivity  illustrate  increasing  numbers  of  carpal 
tunnel  syndrome  and  repetitive  stress  injuries  in  both  the  United  States  and  the  United 
Kingdom  (Mallory  and  Bradford,  1989;  Hill,  1989).  This  reportedly  has  led  some  companies 
to  spend  millions  of  dollars  on  medical  treatment,  ergonomically  correct  equipment,  training 
and  legal  expenses  (Minter,  1989). 

Because  of  the  complexity  of  these  disorders,  however,  and  the  methodological  limitations  of 
the  studies  investigating  them,  their  etiology  cannot  be  attributed  solely  to  the  visual  display 
workstation  environment  (Sauter,  et.  al.,  1991).  Herman  Miller  Research  and  Design  (1991) 
corroborate  the  complex  nature  of  CTDs  by  organizing  risk  factors  into  three  general  groups: 
(I)  ergonomic  stresses;  (2)  psychosocial  stresses;  and  (3)  physiological  predispositions.  The 
dynamic  interrelationships  among  these  factors  is  brought  home  again  by  a  recent  report  by 
NIOSH  (Swaboda,  1992)  showing  that  psychological  factors  such  as  job  insecurity,  work 
demands  and  electronic  monitoring  contribute  to  musculoskeletal  disorders  among  VDT  users 
even  when  preventive  ergonomic  measures  are  implemented.  Etiology  is  further  complicated 
by  chronic  pain  syndromes  (Kusack,  1990).  Therefore,  caution  must  be  used  in  attributing 
the  causes  of  such  injuries.  Nevertheless,  adequate  research  evidence  exists  to  justify  a 
systematic  examination  of  the  ergonomic  aspects  of  work  environments  in  which  mechanical 
input  devices,  such  as  keyboards  (QWERTY),  mice  and  trackballs  are  in  use.  Each  of  these 
devices  forces  users  to  make  repetitive,  small  moveti^.ents  that  over  time  can  cause  RSIs 
(Hurty,  1992). 

A  major  comprehensive  study  of  workstation  design  and  musculoskeletal  discomfort  within 
the  context  of  a  visual  display  terminal  data  entry  task  was  conducted  by  Sauter  and  others 
(1991).  The  primary  objective  of  this  study  was  to  clarify  the  contribution  of  workplace 
ergonomic  factors  to  musculoskeletal  problems  among  VDT  users.  The  study  objective  had 
special  significance  given  statistical  evidence  pointing  to  the  increased  incidence  of 
musculoskeletal  disabilities  among  VDT  workers  in  the  United  States  (Eisen  and  LeGrande, 
1989;  Pasternak,  1989;  NIOSH,  1990).  Study  methodology  attempted  to  overcome  limitations 
of  previous  studies  which  led  to  discrepant  findings  of  the  association  of  ergonomic  factors 
and  musculoskeletal  problems.  Emphasis  was  placed  on:  (1)  objective  assessment  of  work 
postures;  (2)  reclassification  of  continuous  data  on  working  conditions  and  discomfort  into 
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categorical  variables  for  analysis;  (3)  use  of  multivariate  statistical  techniques;  and  (4) 
adjusting  for  biodemographic  factors. 

The  study  findings  provide  support  for  the  emerging  data  on  outbreaks  of  musculoskeletal 
disorders  among  VDT  users  that  point  to  the  vulnerability  of  upper  extremities-the  hand  and 
the  wrist  in  particular.  The  authors  concluded  that  the  study  findings:  (1)  served  to  reduce 
the  uncertainty  regarding  the  contribution  of  ergonomic  factors  to  the  etiology  of 
musculoskeletal  problems;  (2)  suggested  workstation  configurations  that  will  minimize  these 
problems;  and  (3)  improved  the  definition  of  the  regional  pattern  of  musculoskeletal  problems 
among  VDT  users.  Among  the  suggestions  for  environment  and  workstation  design  were: 
(1)  low  placement  of  the  keyboard;  (2)  avoidance  of  low  compressible  seats  to  prevent  leg 
discomfort;  (3)  preference  for  an  erect  sitting  posture  and  backrest  height  adjustments;  and 
(4)  physical  exercise  during  work  and  frequent  rest  breaks. 

Most  of  the  general  information  articles  citing  musculoskeletal  disorders  in  VDT  workstation 
environments  focus  on  carpal  tunnel;  however,  other  musculoskeletal  problems  are  also 
associated  with  VDT  workstations,  A  comprehensive  list  would  include  such  disorders  as 
tendinitis,  tenosynovitis,  trigger  finger,  gamekeeper's  thumb,  Guyon's  canal  syndrome, 
epicondylitis,  cubital  tunnel  syndrome,  low  back  pain,  and  others  (Herman  Miller,  Inc. 
1991).  What  these  disorders  appear  to  have  in  common  is  a  work  environment  characterized 
by:  (1)  awkward  positions;  (2)  localized  pressure;  (3)  holding  a  static  position  without 
movement;  (4)  excessive  use  of  force  or  strength;  and  (5)  repetition  without  rest  breaks 
(Herman  Miller  Research  and  Design,  1991). 

Several  studies  cited  in  Herman  Miller  (1991),  associate  awkward  wrist  positions  with  CTDs. 
Users  of  keyboards,  computer  mice  and  trackballs  are  susceptible  to  awkward  wrist 
positions.  Keyboard  computing-related  CTDs  are  the  result  of  pressure  changes  in  the  carpal 
tunnel  caused  by  many  repetitive  movements  made  while  the  wrist  is  vertically  deviated 
outside  of  neutral  range-either  in  a  position  of  flexion  (bent  such  that  the  palm  is  lower  than 
the  wrist  joint),  or  extension  (bent  such  that  the  palm  is  higher  than  the  wrist  joint)  (Marras 
and  Schoenmarken,  1991).  In  this  case  the  major  postural  risk  factors  are  wrist 
extension/flexion  and  the  acceleration  of  extension/flexion  hand  movements  (Proformix- 
Research  Update  #(X)2,  1991). 

To  minimize  this  risk  and  practice  safe  keyboarding,  it  is  universally  recommended  that 
wrists  be  properly  supported  in  a  neutral  position  with  minimal  flexion  and  extension. 
Alternatives  to  the  flat  keyboard  might  also  provide  a  solution.  Several  alternative  keyboards 
split  the  keys  into  left-hand  and  right-hand  groups,  aiming  them  inward  to  reduce  bent  wrists. 
Others  crack  the  keyboard  in  half,  creating  a  tent  shape  eliminating  the  forearm  rotation 
required  for  standard  keyboards*  Unfortunately,  these  alternatives  are  presently  prototypes 
and  are  not  available  (Buesen,  1984;  Hobday,  1988;  Zipp,  1981). 

Mice  and  trackball  devices  are  beginning  to  attract  the  attention  of  ergonomists.  Since  these 
devices  involve  small,  repetitive  movements  that  have  a  cumulative  effect,  they  also  are 
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looked  upon  as  a  source  of  repetitive  strain  injuries.  Given  their  recent  introduction 
compared  to  the  computer  keyboard,  however,  there  is  scant  ergonomic  information 
(Whitefield,  1989;  Herman  Miller,  1991;  Hurty,  1991).  Because  many  mouse  users  rest 
their  forearms  on  their  work  surface  and  move  the  mouse  around  through  wrist  movements 
they  create  a  side-to-side  bending  of  the  wrists  which  can  potentially  cause  CTDs.  In  the  case 
of  the  trackball,  the  role  of  the  thumb  is  an  important  consideration.  The  thumb's  limited 
freedom  of  movement  is  well  suited  for  side-to-side  grasping  actions  but  makes  it  a  poor 
choice  for  controlling  up-and-down  or  circular  motions  (Hurty,  1991).  Touch  screens  also 
deserve  attention.  Touch  screens  are  appealing  input  devices  for  a  number  of  reasons:  (1) 
users  point  directly  at  an  object;  (2)  they  require  little  or  no  training;  (3)  they  are  faster  than 
other  pointing  devices;  and  (4)  no  extra  work  surface  is  required.  The  down  side  to  touch 
screens  involves  arm  fatigue,  callouses  on  fingers  with  heavy  use,  smudges,  optical 
interference  and  increased  glare. 

With  all  these  devices,  as  well  as  with  the  keyboard,  it  is  important  to  address  the  broader 
issue  of  overall  workstation  design  to  ensure  prevention  of  musculoskeletal  injuries.  In  as 
much  as  musculoskeletal  injuries  are  influenced  by  workstation  design  and  the 
interrelationships  among  psychological  stresses  and  physiological  predispositions,  it  is  crucial 
that  studies  on  etiology  be  implemented  as  controlled  experiments.  These  studies  should  have 
valid  experimental  methodologies  and  should  be  foUowed-up  by  case  control  and  prospective 
studies  performed  on  ergonomic  interventions  in  actual  work  settings  (Sauter,  1991), 

Lab  and  field  studies  should  also  be  developed  to  provide  practical  data  on  specific  ways  in 
which  preferred  individual  adjustments  of  workstations  differ  from  the  recommended 
standards.  Some  recent  studies  have  shown  that  individuals  prefer  adjustments  that  differ 
from  the  standards  (Jaschinski,  1988;  Sautet,  1991).  Standards  are  not  meant  to  be 
monolithic.  Professional  ergonomists  and  users  should  work  together  towards  the  goal  of 
developing  equipment  and  workstations  that  allow  the  user  maximum  flexibility  to  customize 
the  configuration. 

VISUAL  DISPLAY  TERMINALS  AND  VISION  COIVIPLAINTS 

In  a  recent  survey  conducted  by  the  American  Optometric  Association,  eyestrain,  headaches, 
and  blurred  vision  were  ranked  as  the  top  three  vision  complaints  associated  with  the  use  of 
VDTs  (Sheedy,  1992).  Visual  fatigue  and  double  vision  have  been  reported  by  other  surveys. 
Studies  are  divided,  however,  on  the  issue  of  whether  VDT  use  directly  causes  increased 
visual  problems. 

Research  distinguishes  between  visual  causative  factors  (specific  diagnosable  visual  disorders 
that  an  individual  has)  and  environmental  causative  factors  (specific  factors  in  the  design  of 
the  VDT,  the  workstation  and  the  environment)  (Sheedy  and  Parsons,  1990).  It  must  be 
noted  also,  that  an  individual's  visual  problems  may  be  due  to  a  single  cause  or  a 
combination  of  causes.  Since  it  is  duly  noted  that:  (a)  individual  predispositions  have  an 
effect  on  visual  problems;  (b)  problems  can  have  one  or  more  causes;  and  (c)  separating  out 
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predispositions  from  environmental  factors  is  a  daunting  prospect,  it  will  suffice  to  simply 
delineate  some  of  the  environmental  factors  that  are  amenable  to  change  and  that  may 
influence  vision  complaints. 

The  major  environmental  problem  involves  lighting.  Florescent  overhead  lighting  is  the 
norm  in  most  offices.  This  results  in  100-150  ft.  candles  of  illumination  on  desk  surfaces. 
This  level  greatly  exceeds  the  18-46  ft.  candles  recommended  by  the  ANSI  Standard  (1988). 
Because  desk  and  reference  documents  are  significantly  brighter  than  the  VDT  screen, 
adaptation  problems  may  occur  as  the  result  of  the  users  having  to  move  their  eyes  back  and 
forth  from  brightly  lit  documents  to  the  screen.  Another  source  of  eye  discomfort  is  glare 
off  the  screen  caused  by  bright  lights  in  peripheral  vision  (either  from  florescent  lights  or 
bright  outside  light).  These  problems  can  be  alleviated  by  turning  off  some  overhead  lights, 
drawing  blinds  or  shades  or  moving  the  VDT,  perhaps  recessing  it  so  that  it  is  lower  than  the 
desk  surface.  Reflections  on  the  screen  can  also  interfere  with  resolution  and  contrast.  Glass 
anti-reflection  screens  are  a  good  solution  and  are  preferred  over  mesh  screens.  VDTs  with 
dark  characters  on  white  background  provide  better  resolution.  A  screen  refresh  rate  of  at 
least  60  Hz  (German  health  regulations  require  70  Hz)  is  preferred.  A  lower  refresh  rate 
creates  screen  flicker  which  may  cause  eyestrain  and  headaches.  For  optimal  viewing  of  the 
screen,  the  screen  height  should  be  adjusted  to  10-20°  below  the  user's  eyes.  This  range 
appears  to  alleviate  neck  and  back  ache  as  well  as  dry  irritated  eyes  and  contact  lens 
difficulties  because  of  decreased  blinking  (Sheedy,  1992). 

Adult  learners  working  with  VDTs  are  faced  with  a  special  problem.  The  loss  of  eye 
focusing  that  invariably  accompanies  advancing  age  (presbyopia)  requires  special  prescription 
lenses  for  the  adult  population  (Sheedy,  1992).  In  the  Sheedy  (1992)  survey,  thirty  percent 
of  all  VDT  patients  received  prescriptions  for  bifocals  and  nineteen  percent  for  trifocals. 
Bifocal  and  trifocal  users  may  experience  neck  and  back  problems  in  addition  to  vision 
complaints  as  they  attempt  to  accommodate  two  or  three  visual  focal  points.  The  author  can 
attest  to  this  special  difficulty,  having  word  processed  this  paper  while  wearing  trifocals. 

The  American  Optometric  Association  estimates  that  among  the  seventy  million  eye  exams  a 
year,  ten  million  patients  obtain  eye  exams  because  of  problems  with  VDTs  (Sheedy  1992). 
Thirty-seven  percent  of  these  complaints  are  attributed  to  environmental  visual  factors  such 
as  lighting,  poor  screen  resolution  and  glare.  Neck  and  back  ache  are  also  reported  with 
these  vision  complaints  due  to  high  VDT  placement. 

VISUAL  DISPLAY  TERMINALS  AND  PSYCHOSOCIAL  STRESSES 

The  recent  NIOSH  study  (Swoboda,  1992)  linking  job  stress  to  VDT  physical  injuries 
reinforces  the  ergonomic  perspective  of  the  dynamic  interplay  among  the  worker,  the  job,  the 
workstation,  and  the  environment.  As  reported  in  the  Washington  Post  (July  21,  1992)  the 
NIOSH  study  was  conducted  at  the  request  of  US  West,  Inc.  and  the  Communications 
Workers  of  America  Union.  The  study  reported  that  musculoskeletal  disorders  occurred 
among  VDT  workers  even  in  situations  where  ergonomic  precautions  where  implemented  by 
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employers  to  protect  them.  Even  when  preventive  workstation  designs  were  implemented, 
psychosocial  factors  apparently  influenced  the  development  of  musculoskeletal  injuries.  This 
finding  has  caused  leaders  at  US  West  to  view  management's  treatment  of  workers  as  equally 
important  to  an  injury  free  environment  as  the  equipment  and  the  workstation. 

Earlier  studies  (Sauter,  1981;  Smith,  1983;  Amdt,  1984;  Sauter,  1984)  reported  on  the 
general  stress  levels  caused  by  the  introduction  of  automated  technology  in  the  office; 
specifically,  the  visual  display  terminal.  These  studies  did  not  look  for  the  effect  of 
psychosocial  (organizational  or  social  environment)  stresses  on  physical  complaints  of  VDT 
users.  Like  the  previous  subjects  discussed,  it  is  difficult  to  demonstrate  a  cause  and  effect 
relationship  from  the  research.  However,  it  is  informative  to  review  the  results  of  a  major 
ergonomic  intervention  study  conducted  at  Federal  Express  Company. 

A  report  of  this  ergonomic  intervention  was  presented  as  a  case  study  by  Allen  Westin  in 
Promoting  Health  and  Productivity  in  the  Computerized  Office:  Models  of  Successful 
Ergonomic  Interventions  (1990).  By  1983,  Federal  Express  had  8000  VDTs  in  use 
throughout  the  organization.  Federal  Express's  Director  of  Human  Resources  Analysis 
established  the  proactive  view  that  the  issue  of  VDT  use  was  not  a  narrow  ergonomics  issue 
involving  only  equipment  and  workstations.  He  expanded  ergonomics  to  include  job  design, 
work-group  relations,  employee  education  and  manager  training.  Through  surveys  and 
interviews,  management  staff  discovered  that  employees  perceived  a  direct  relationship 
between  job  stress,  VDT  work,  and  their  health.  With  this  information.  Federal  Express 
developed  a  "corporate  ergonomic  standard"  to  ensure  the  health  and  well-being  of  those 
whose  work  environment  involved  video  display  terminal  use.  In  1990,  Federal  Express  had 
52,300  VDTs  and  an  additional  3000-4000  stand-alone  PCs. 

Tlie  pay  off  from  this  forward-looking  approach  has  been  no  reported  cases  of  repetitive 
strain  injury  at  Federal  Express  despite  their  intensive  and  extensive  VDT  use.  There  have 
been  fewer  than  six  stress  related  claims  by  VDT  users  filed  under  worker  compensation  in 
the  past  five  years.  The  model  implemented  at  Federal  Express  should  be  reviewed  by  other 
companies  or  groups  as  a  potential  approach  for  preventing  psychosocial  stresses  from 
contributing  to  the  development  of  musculoskeletal  disorders  among  VDT  users. 

Other  ergonomic  interventions  have  reported  similar  positive  results  (Sauter,  et  al.  1990). 
Ergonomics  in  these  interventions  was  broadly  defined,  as  in  the  Federal  Express  case,  to 
include:  (1)  equipment  and  workstation  design;  (2)  job  design;  (3)  work-group  relations;  and 
(4)  education  of  employees  and  training  of  staff  The  interventions  reported  show  a  matrix  of 
effects  suggesting  significant  health  and  productivity  gains  from  the  effective  application  of 
ergonomics  in  the  VDT  workplace.  These  studies  also  demonstrate  dramatic  productivity 
increases  in  the  range  of  twenty  percent  resulting  from  ergonomic  improvements  in  the 
design  of  job,  and  the  workplace.  A  concomitant  increase  in  worker  satisfaction  and  a 
decrease  in  somatic  complaints  were  also  reported.  Absenteeism,  turnover,  sick  leave  and 
compensation  claims  were  also  in  decline. 
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CONCLUSION 

New  interactive  technologies  present  a  challenge  and  an  opportunity  for  designers  of  learning 
environments.  This  paper  has  touched  upon  several  of  the  major  variables  that  influence  the 
design  of  safe  and  effective  visual  display  learning  environments.  An  ergonomic  model  was 
presented  as  an  organizing  concept  for  defining  the  domain  of  ergonomic  variables  relevant 
to  the  study  of  learning  environments.  Research  on  ergonomics  in  the  workplace  was 
reviewed  with  the  goal  of  deriving  relevant  guidelines  for  designing  learning  environments 
for  interactive  technologies.  Based  on  this  review,  and  the  perusal  of  major  documents  from 
science  and  industry  on  ergonomic  standards  and  specifications,  a  set  of  guidelines  was 
presented.  It  is  hoped  that  these  guidelines  will  be  useful  to  health  science  educators  and 
administrators. 

The  application  of  ergonomics  is  a  complex  process  that  requires  careful  management.  To 
ensure  that  standards  and  guidelines  drawn  from  human  factors  and  anthropometric  data  are 
safe  and  effective  for  individuals,  they  must  be  validated  through  prospective  demonstrations 
and  case  studies  in  "real  life"  learning  environments. 

GUIDELINES  FOR  AN  ERGONOMICALLY  DESIGNED  WORKSTATION 

General  Workstation  Specifications 

o        Worksurface  depth  and  width-a  minimum  depth  of  30  inches  and  width  of  48  inches- 
surface  space  must  allow  for  efficient  organization  of  documents,  papers  and  other 
materials  relevant  to  the  learning  task 

o       Worksurface  height-if  adjustable,  between  25.5  and  29.5  inches  for  effective  use  of 
keyboard  and  mouse-if  not  adjustable,  28.3  inches  as  a  compromise-option  of  the 
keyboard  tray  below  the  work  surface  which  places  hands  and  wrists  in  a  wrist  neutral 
positiofi 

0       Surfaces  should  have  buUnose  edges  to  avoid  sharp  comers 

0       The  keying  surface  between  23  inches  and  28.5  inches  from  the  floor  with  detachable 
keyboard-top  surface  of  the  "home  row"  of  keys  no  higher  than  2-2.5  inches  above 
worksurface-to  allow  elbows  at  90°  and  arms  and  hands  parallel  to  the  floor- 
keyboard  where  thumb  joints  are  located-wrist  rests  are  helpful 

0        A  mouse  and  writing  platform  that  are  positioned  within  the  primary  reach  zone  and 
will  give  hand/wrist  support  during  use 

0       Research  findings  on  carpal  tunnel  suggest  options  of-  an  adjustable  keyboard  tray 

below  the  work  surface  which  places  hands  and  wrists  in  a  "wrist  neutral"  position  or 
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the  Tony  Keyboard  that  is  hinged  in  the  middle  reducing  pressure  on  the  median 
nerve 

0        Interactive  and  multimedia  equipment  requires  additional  space  for  peripherals  such  as 
modems;  CD-ROM  drives;  laser  videodiscs;  second  monitor;  and  printers-a  plan  for 
how  equipment  will  be  used  will  determine  space  requirements-options  include 
stacking  on  surface  or  additional  adjacent,  above  or  below  moveable  surfaces 

Visual  DispSay 

o       Distance  between  learner's  eyes  and  visual  display  should  be  between  18-28  inches  to 
provide  preventive  measures  against  the  potential  hazards  from  extremely  low 
frequency  electromagnetic  emissions  (ELF)-Swedish  National  Board  for  Measurement 
and  Testing  g:  Melines  (MPR-2)  and  IEEE  recommendations-restrict  ELF  magnetic 
fields  (5Hz-2kHz)  to  an  intensity  of  no  more  than  2.5mg  (milligauss)  at  half  a  meter 
(19.7  inches)  from  the  monitor 

0        Acceptable  viewing  angles  range  from  0-60^  from  the  top  of  the  screen  to  the  home 
row  of  the  keyboard-preferred  viewing  angle  is  somewhere  between  10-20^  below  the 
horizontal-display  should  be  located  so  that  the  individual's  normal  line  of  vision  falls 
in  the  upper  half  of  the  display 

o        Monitor  should  have  independent  height  and  angle  tilt  and  swivel  adjustments-Glass 
anti-glare  screens  are  preferred  over  mesh  screens-special  glare  control  screen  filters 
are  available  to  reduce  staiic  electricity 

o        Monitors  should  be  placed  at  90^  angle  to  windows  and  never  directly  in  front  of 
windows 

0        Display  should  be  "flicker  free" -refresh  rate  of  at  least  60Hz  and  higher  are 
preferred  (German  standard  is  72Hz) 

o        Non-interlaced  monitors  are  preferred  over  interlaced 

o       Keep  monitors  clean 

Seating  (Cited  primarily  from  ANSI/HSF  100-1988) 

o        Seat  height-consensus  not  available  on  optimal  seat  height-ANSI/HSF  100-1988  calls 
for  seat  height  adjustment  of  16  to  20.5  inches-Herman  Miller  et.  al.,  show  that  40% 
of  subjects  set  their  chairs  at  20.25  inches  or  higher-difference  points  up  need  for 
adjustable  chair 
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o       Seat  depth  should  be  between  15  and  17  inches-it  is  recommended  that  seats  have  a 
"waterfall"  contour  just  behind  the  user's  knee  and  underside  of  the  thigh  to  avoid 
excessive  pressure  in  this  area 

o        Seat  width  minimum  is  set  at  18.2  inches 

o       Seat  pan  angle  when  user  has  feet  flat  on  floor  must  ensure  angle  between  upper  and 
lower  leg  of  between  60  and  100° 

o       Seat  pan  and  pan  angle  should  range  from  90  to  105° 

o  A  seat  back  rest  and  lumbar  support  should  be  provided-no  specific  guidelines  given 
Acoustics,  Lighting  and  General  Environment 

0        Acoustic  panels  are  required  for  workstations  in  addition  to  acoustically  treated 
ceilings-goal  is  to  balance  sound  from  background  and  adjacent  sources 

0       Ambient  or  background  noise  (white  noise)  is  desirable  at  a  level  that  does  not 
interfere  with  task  performance  (between  40-55dba) 

o        Illuminance  sources  should  be  designed  and  located  to  minimize  glare— illuminance 
measured  in  the  range  of  200  lux  to  500  lux  on  the  work  area  is  sufficient 

o        Task  or  local  lighting  may  be  needed  for  reading  documents  or  passive  light- 
absorbing  displays  with  inadequate  luminance-task  lights  with  batwing  lenses  provide 
sufficient  light  and  minimize  glare 

0        A  combination  of  indirect  -overhead,  natural  and  task  lighting  is  preferable 

0        For  close-up  work  light  should  be  directed  sideways  onto  documents  to  avoid 
producing  glare  on  monitor-helps  prevent  eye  fatigue  and  headaches 

0       Florescent  glare  can  be  reduced  by  replacing  plastic  prism  coverings  with  grids  that 
break  up  the  light  pattern 

0       Color  scheme  should  be  neutral  and  pleasing  to  the  eye 

0        Proper  ventilation  requires  about  two  air  exchanges  per  hour-temperature  and 
humidity  should  be  kept  constant  with  temperature  range  from  68  to  75^  F 
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Multiple  Group  Learning  Workstations 

o       Learning  stations  should  be  configured  so  that  no  individual  ^omes  within  48  inches 
of  the  side  or  back  of  a  monitor-comer  arrangement  prevents  extremely  low- 
frequency  (ELF)  problems~when  users  face  each  other,  check  distance  from  back  of 
monitor  to  opposite  user 

o       Power  supply  under  a  raised  floor  with  4-plex  grounded  outlets  every  five  feet 

o       Information  on  electrical  requirements  (watts)  per  computer  and  peripheral  type 
should  be  calculated  based  on  required  configurations-information  is  available  in 
technical  specifications  from  manufacturers 

o       Cables,  wires,  and  electrical  cords  should  be  channeled  with  a  wire  management 
system  built  into  the  workstation  panels 

0       Separate  air  conditioning  for  workstation  area  with  circuits  separate  from  other  areas 
and  no  mechanical  devices 

o        For  small  group  work—  a  viewing  area  adjacent  to  workstations  equipped  with  wall 
screen  and  multiscan  video  projection  system  for  accommodating  different  monitor 
scanning  rates 

o       Small  group  area  convertible  into  break  area  with  tables  and  comfortable  chairs  for 
relaxation,  stretching  and  change  of  activities 

Psychosocial  Harmony  Tlirough  Training  and  Exercise 

o       Teachers  should  receive  orientation  and  training  with  regard  to  their  perceptions  of 
how  the  technology  will  be  used  and  their  expectations  of  the  learners'  behaviors- 
these  expectations  should  be  clearly  communicated  to  the  learners  and  the  learners 
should  have  an  opportunity  to  acknowledge  them 

o       Learners  should  receive  proper  orientation  and  training  with  regard  to  what  behaviors 
are  expected  of  them  in  this  environment  and  in  addition,  they  must  receive  specific 
instructions  on  how  to  operate  all  equipment  and  on  how  to  effectively  customize  the 
adjustable  features  of  their  environment 

o       Adequate  opportunities  should  be  given  for  rest  breaks  and  exercise-in  addition  to 
brief  exercises  for  flexibility,  stretching  and  strength  development,  vision  aerobics 
software  is  available  for  depth  perception,  focusing  skills,  frequent  blinking  and 
relaxation  exercises 
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Development  of  an  Interactive  Multimedia  System  Presenting 
Food  Safety  to  Navajo  Audiences 

Jeanne  Gleason,  New  Mexico  State  University 

Walk  in  Beauty.  Walk  in  Danger  is  a  touch  screen  interactive  multimedia  system  presenting  safe 
food  storage  and  handling  practices  to  members  of  the  Navajo  Nation.  Each  system  includes 
a  Macintosh  computer,  customized  laserdisc  taped  on  the  Navajo  Reservation,  and  computer 
graphics. 

Communications  research  indicates  the  users  ability  to  control  the  pace  of  the  presentation  is 
particularly  appropriate  for  Navajos  whose  communications  style  incorporates  pauses  or  silence 
uncharacteristic  of  English  speakers  and  who  are  reluctant  to  talk  with  those  who  are  not  close 
acquaintances- 

This  presentation  will  cover: 

—artistic  designs  for  Navajo  audiences, 

-impacts  of  Navajo  customs  and  spiritual  beliefs, 

-pacing  and  design  of  video  clips, 

—use  of  Navajo  and  English  audio, 

-menu  designs  and  user  options, 

—information  design  for  vaiied  audiences, 

—installation  and  maintenance  logistics,  and 

-formative  and  summative  evaluation. 

The  presenter  will  also  cover  how  to  deal  with  an  advisory  committee  and  target  audience  400 
miles  away,  and  incorporate  students  into  the  development  process. 

Development  is  funded  by  the  W.K.  Kellogg  Foundation,  the  United  States  Department  of 
Agriculture,  and  the  New  Mexico  Cooperative  Extension  Service.  Topics  covered  include  safe 
food  storage  and  food  handling,  home  butchering,  cleanliness  with  limited  water,  use  of 
commodity  foods,  and  food  safety  at  tribal  gatherings. 
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CAD:  AN  EXPRESSIVE  PARTNER,  A  CHALLENGE,  A  VIOLIN! 


Stqphanie  A.  Clemons,  M.S.,  Colorado  State  University 
Lisa  Waxman,  Florida  State  University 
Diane  Davis,  Valparaiso  University 


Abstract 

The  first  purpose  of  this  paper  is  to  describe  three  specific  assignments  which  use  CAD  to  document 
the  design  process  or  use  CAD  as  a  design  tool  throughout  tlie  design  process.  The  second  purpose 
is  to  address  common  challenges  design  educators  may  face  as  they  begin,  or  continue,  to  integrate 
CAD  into  their  curriculum.  Care  was  taken  to  represent  three  different  design  programs  across  the 
nation,  as  well  as  various  computer  platforms  and  software  techniques. 


Introduction 

Throughout  time,  people  have  felt  the  need  to  express  and  share  their  ideas.  The  advent  of  the 
printing  press  greatly  expanded  people's  ability  to  share  their  thoughts,  yet  the  technology  of  the  press 
remained  secondary  to  the  expression  of  ideas.  Today,  computers  in  general,  and  computer  aided 
design  (CAD)  systems  specifically,  seem  to  dominate  as  a  technical  device  rather  than  as  a  tool  for 
expression.  There  is  a  widespread  stereotype  of  computers  as  machines  with  which  people  can  only 
form  an  analytical,  rule-based  relationship  (Turkle,  1991).  Recognizing  the  computer's  second  nature 
as  an  expressive  medium  is  important  for  design  educators  especially  since  CAD  systems  have  become 
incorporated  into  areas  typically  dominated  by  designers.  CAD  systems,  with  the  new  graphic 
software,  offers  educators  an  opportunity  to  assist  the  students  in  developing  a  more  subjective,  rather 
than  objective,  relationship  with  computers. 

The  role  of  CAD,  as  a  technical  device,  has  been  fostered  in  interior  design  because  frequently  the 
initial  users  of  this  technology  have  been  technicians,  not  designers.  In  many  work  settings,  designers 
typically  created  and  the  technicians  entered  the  results  into  the  CAD  system.  Computer  instruction 
^so  fostered  this  role.  Often  the  goal  was  to  teach  systems,  platforms,  software-hence  the 
technology— rather  than  teach  design  applications  using  the  computer  as  a  tool.  Our  language  of 
"tools"  typically  revolves  around  the  "best"  system,  the  most  "efficient"  system,  the  most  "user 
friendly"  system.  Current  textbooks  are  written  as  a  self-paced  tutorials  where  the  student  is  given 
step-by-step  instructions  on  replicating  a  provided  design  solution  (McLain-Kark,  1991;  Wohlers, 
1991).  For  computers  to  become  a  design  tool,  they  must  be  incorporated  into  the  design  process  with 
students  producing  their  own  ideas. 
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Background 


CAD  has  becoire  an  integral  part  of  interior  design  education  (McLain-Kark  and  Rawls,  1988). 
Research  has  si  own  there  is  a  relationship  between  familiarity  with  CAD  and  fluency  of  ideas 
(Eshelman  &  Vi  legas,  1990).  Interior  design  programs  are  struggling  with  the  integration  of  CAD 
into  design  studios  (Inman,  et  al,  1990).  The  general  use  of  the  computer,  as  a  design  tool,  is  starting 
to  be  used  in  the  design  process  in  sophomore  studio  classes  (Jones,  1991).  Results  from  a  survey  of 
practitioners  revealed  that  interior  designers  strongly  believe  that  students  should  have  CAD  instruction 
prior  to  graduation  (Clemons  &  McCuUough,  1989).  Educators  have  responded  to  this  chsdlenge. 
Research  indicates  that  almost  90  percent  of  the  interior  design  programs  with  IDEC  members  are 
offering  optional  or  required  courses  on  CAD.  In  addition,  interior  design  faculty  are  providing  this 
instruction  (Clemons,  1991).  The  1990's  could  be  called  the  "second  generation"  of  computers  in 
design  education  with  CAD  no  longer  used  solely  as  a  technical  device,  but  incorporated  into  studios 
(ACADIA,  1986).  With  this  ever-changing  technology,  new  problems  are  cropping  up  to  challenge 
the  time,  patience,  and  skills  of  the  design  educator. 


Purpose 

The  goals  of  this  paper  are  twofold.  The  first  purpose  is  to  describe  how  three  specific  interior  design 
projects  are  being  used  to  document  the  design  process  or  use  CAD  as  a  design  tool  throughout  the 
design  process.  The  second  purpose  is  to  address  common  challenges  design  educators  may  face  as 
they  begin,  or  continue,  to  integrate  CAD  into  their  curriculum.  Care  was  taken  to  represent  three 
different  design  programs  across  the  nation,  as  well  as  various  computer  platforms  and  software 
techniques. 


Design  Process 

The  objectives  of  the  first  project,  an  introductory  assignment,  was  to  give  students  a  chance  to  design 
a  two-dimensional  project  using  CAD  as  an  expressive  tool.  Each  student  designed  one  of  the 
following:  entry  way /door  facade,  fireplace/mantel  design,  piece  of  furniture,  custom  area  rug,  or 
custom  fabric  design.  Two  alternatives  to  their  design  were  then  presented.  The  process  of 
developing  a  second  design,  by  modifying  the  first,  encouraged  students  to  explore  various 
possibilities.  Many  times  the  second  designs  were  stronger  than  the  first.  Seeking  out  design 
solutions  was  simplified  with  the  use  of  CAD.  Students  easily  made  changes  and  the  habit  of  looking 
for  further  solutions  enhanced  their  problem  solving  skills.  These  skills  carried  over  into  more 
advanced  CAD  projects,  as  well  as  other  studio  classes. 

The  second  project  was  an  assignment  for  a  resort  hotel  design  (IFDA  Student  Competition,  1991). 
The  objectives  of  this  project  were  to  have  students  develop  a  comprehensive  image  package  based 
on  an  individual  concept.  The  project  program  included  designing  a  logo,  signage  package,  hotel 
amenities  with  logo,  uniforms  and  table  place  settings  including  dishes,  flatware  and  linens.  Students 


160 

160 


3 

gained  experience  using  an  electronic  garment  library,  electronic  color,  as  well  as  graphics  and  word 
processing  programs.  The  project  was  very  successful  in  designing  and  applying  an  image  to  a  variety 
of  objects.  The  development  and  manipulation  of  color  resulted  in  more  refined  color  schemes  on  the 
computer  and  more  accurately  rendered  color  off  of  the  computer.  Two  of  the  students  were  regional 
competition  winners  and  one  received  a  national  honorable  mention. 

The  third  project  sought  to  document  the  design  process  using  a  CAD  system.  Students  were  given 
a  design  problem  and  were  instructed  to  document  their  design  process  by  using  the  "make  slide  and 
view  slide**  command  within  AutoCAD  software.  The  generated  computer  slides  record  the  process 
as  the  student's  design  solution  evolved.  A  script  file,  or  mini  program,  which  nests  the  slides 
together  in  an  animated  documentation  is  then  created.  Other  possible  applications  of  the  script  file 
might  be  to  record  all  of  the  students  drawings  in  an  individual  ''animated  portfolio"  or  record  student 
work  completed  in  a  semester  or  year  for  marketing  and  accreditation  purposes. 


Common  Challenges 

The  second  goal  of  this  paper  is  to  address  common  challenges  interior  design  educators  may  face  as 
they  begin,  or  continue,  to  integrate  CAD  into  their  curriculum.  Educators  may  have  to  cope  with 
the  reliance  on  computer  technicians,  providing  lab  hours  for  students,  working  with  administrators 
on  behalf  of  the  CAD  program,  modifying  teaching  techniques,  and  educating  peers.  These  hints 
should  be  of  interest  to  educators  who  wish  to  avoid  the  areas  of  stress,  frustration  and  time 
consuming  activities  possible  with  the  further  implementation  of  CAD.  Discussion  of  these  common 
difficulties  may  make  other  educators  aware  their  problems  are  not  isolated  to  their  class,  program  or 
school  and  will  offer  possible  solutions. 

Reliance  on  a  Technician 

Historically,  manual  tools  such  as  the  T-square,  drafting  table  and  pencil  were  tools  to  teach  drafting, 
space  planning  and  communication  skills  to  the  student.  Today,  few  educators  consider  themselves 
"computer  experts"  with  the  capabilities  to  solve  problems  inherent  to  CAD  systems.  A  dependency 
on  the  technical  expertise  of  computer  technicians  is  adding  a  new  dimension  to  developing  and 
teaching  courses  in  which  the  computer  is  a  tool.  Establishing  course  goals  and  creating  an 
environment  for  learning  requires  foresight  and,  many  times,  compromise.  Time  conflicts, 
communication,  and  awareness  of  each  other's  responsibilities  are  challenges  for  both  the  educator  and 
the  technician  to  address. 

Lab  Hours 

Rarely  do  design  programs  have  money  to  establish  and  maintain  a  CAD  lab  for  their  exclusive  use. 
Typically,  the  labs  are  shared  areas  between  programs,  departments  and  colleges.  Because  the  expense 
of  the  lab  needs  to  be  justified,  the  classroom  is  used  for  multiple  courses.  The  result  is  few  available 
hours  for  students  to  complete  their  homework  assignments.    Solutions  some  universities  have 
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incorporated  are  combination  locks  placed  on  the  lab  doors  for  students  to  use  or  have  provided  keys 
that  students  are  allowed  to  check  out  overnight.  Others  have  expanded  existing  lab  hours  through 
departmental  donations  of  student  workstudy  hours  or  graduate  student  hours,  to  extend  the  operation 
of  the  lab. 

Software  and  hardware  technologies  are  advanced  continuously  on  a  yearly  basis.  If  the  design 
program  can  afford  to  purchase  available  updates,  educators  are  teaching  CAD  with  new  software  and 
hardware  each  year.  Who  teaches  the  teacher?  Continuing  education  courses  are  taught  at  community 
colleges.  Correspondence  courses  are  available.  However,  too  often,  the  educator  sits  with  their 
"nose  in  the  manual"  to  stay  one  step  ahead  of  the  student.  Remaining  current  on  software  and 
hardware  is  time  consuming  and  somewhat  frustrating  for  the  design  educator. 

Support  from  Administration 

The  ability  of  administrators,  deans,  department  heads  and  coordinators  to  operate  a  computer  are 
variable.  Some  advocate  and  encourage  design  educators  to  learn  the  newest  technology  (Inman,  et 
al,  1990),  Some  administrators  have  never  used  a  keyboard,  deeming  it  the  secretary's  territory. 
Educating  administrators  on  CAD's  unique  problems  and  capabilities  can  prove  to  be  both  a  joy  and 
a  headache-  Unless  the  education  of  administrators  is  handled  properly  and  support  is  provided,  they 
can  become  a  major  stumbling  block  in  the  acquisition  of  hardware,  software,  and  technical  assistance 
for  teaching  CAD. 

Teaching  Techniques 


Many  students  take  independent  study  courses  on  CAD  that  propel  them  ahead  of  the  educator's 
knowledge.  A  new  paradigm  for  teaching  and  managing  is  developing  on  the  business  and  education 
horizons.  The  new  paradigm  is  based  on  the  premise  that  the  "heroic"  form  of  leadership  is  obsolete, 
and  that  "developmental"  leadership  is  the  more  appropriate  model  for  the  future  (Tubbs,  1989).  CAD 
classes  can  be  a  prime  place  to  practice  developmental  teaching  where  the  teacher  is  not  the  "end  all" 
but  acts  as  a  facilitator  of  the  students'  learning  process.  Student  mentors  in  the  class  can  assist  in 
the  facilitation  and  remove  some  of  the  minute-to-minute  demands  students  may  place  on  the 
instructor.  This  type  of  teaching  style  can  encourage  "design  team"  skills  and  assist  tiiie  student  in 
individual  problem  solving. 

Attity(lg$  of  Pggrs 

Many  times  the  educator  teaching  CAD  is  the  assigned  "computer  expert"  on  the  interior  design  staff. 
They  act  as  a  catalyst  in  spreading  information  on  computer  capabilities  to  other  members. 
Occasionally  there  is  resistance  to  change  from  peers.  Educators  teaching  CAD  find  themselves  with 
unique  problems  inherent  to  technology  and  without  the  emotional  or  pragmatic  support  from  peers. 
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For  those  individuals,  it  is  critical  to  develop  networks  with  educators  at  other  schools  or  with 
technical  support  people  around  campus. 


Summary 

In  summary,  CAD  systems  as  a  tool  should  be  presented  to  students  as  a  paintbrush  or  a  violin  in 
which  a  cultural  design  can  be  developed  and  expressed;  rather  than  a  tool  like  a  hammer  or  chisel, 
that  roughs  out  the  work.  By  developing  assignments  where  the  student  is  involved  in  the  total  design 
process,  a  more  subjective,  enjoyable  relationship  can  be  engendered  for  the  student. 

Practicing  designers  are  expecting  interior  design  graduates  to  have  skills  utilizing  the  CAD  system. 
Historically,  educators  have  relied  on  their  own  skills  to  teaching  drafting  and  design.  With  the 
implementation  of  CAD,  new  pitfalls  are  developing  on  the  educators'  horizon.  By  addressing 
common  challenges,  other  educators  may  avoid  the  areas  of  stress,  frustration,  isolation  and  time 
consuming  activities  possible  with  the  implementation  of  CAD.  Solutions  are  available. 

CAD  hardware,  software  and  peripheral  equipment  is  changing  on  a  yearly  basis.  It  is  difficult  for 
instructors  to  stay  abreast  of  the  new  technological  advancements.  Likewise,  it  will  be  a  challenge 
for  the  students,  upon  graduation,  to  remain  current  on  CAD.  To  support  the  "learning  to  learn" 
premise,  projects  appropriate  for  computerization  need  to  be  identified  within  existing  course  content. 
It  is  vital  that  projects  adapted  for  computer  instruction  allow  for  design  decision  making  and  not  be 
step-by-step  replications  of  a  provided  solution.  It  is  the  opinion  of  the  authors  that  teaching  students 
how  to  use  CAD  during  the  design  process  will  have  a  more  lasting  effect  on  the  student's  education 
than  how  to  replicate  a  completed  design  solution. 
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EFFECTS  OF  PRESENTATION  TECHNIQUES 
ON  CAD  WORKSHOP  PARTICIPANTS'  ATTITUDES 
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Abstract 

This  research  investigated  whether  attitudes,  opinions,  and  behavior  held  by  practicing  interior 
designers  toward  CAD  differed  depending  on  the  exposure  to  a  hands-on  versus  a  lecture-style 
teaching  technique.  Differences  between  pre-  and  post-test  for  the  lecture-style  group  were  found 
significant  for  five  of  the  six  variables  studied.  One  day  hands-on  CAD  workshops  for  those  just 
learning  to  operate  CAD  were  not  significant  in  changing  participants'  attitudes  or  opinions.  The 
learning  cur\'e  may  be  too  difficult  for  a  one-time  exposure.  However,  overall  attitudes  towai'd  CAD 
remained  positive  for  both  groups. 


Introduction 

Computer  usage  is  quite  common  in  interior  design  firms  today.  Research  shows  that  many  benefits 
can  be  gained  from  the  installation  of  a  computer  aided  design  (CAD)  system  (Berghauser,  1986; 
McLain-Kark,  1986).  Favorable  attitudes  of  practicing  interior  designers  toward  CAD  usage  have 
been  shown  to  be  related  to  the  exposure  or  experience  the  designer  had  had  v/ith  CAD.  Past  research 
supports  the  assumption  that  educating  designers  about  CAD  systems  will  facilitate  a  more  rapid 
adoption  of  the  tool  into  the  design  field  (Cieraons  &  McCulIough,  1989). 

Eshelman  and  Villegas  (1990)  performed  a  pilot  study  to  test  the  hypothesis  that  designers  will 
experience  an  increase  in  fluency  of  id^as  as  they  become  more  familiar  with  a  new  tool-CAD.  Their 
hypothesis  was  confirmed  by  the  results.  Prom  a  list  of  extraneous  variables  that  could  have  affected 
the  positive  correlation  between  familiarity  with  CAD  and  fluency  of  different  ideas,  the  attitude  of 
the  participants  towaid  learning  CAD  appeared  to  be  the  most  influential. 

The  purpose  of  this  research  was  to  investigate  (1)  whether  attitudes  and  opinions  held  by  practicing 
interior  designers  toward  CAD  differed  when  exposed  to  a  hands-on  versus  a  lecture-style  teaching 
technique  and  (2)  if  their  behavior  toward  CAD  usage  had  modified  one  year  after  attending  the 
workshop.  A  pre-  and  post-test  design  was  used  to  assess  the  results  of  a  hands-on  versus  lecture-style 
workshop  on  designers'  attitudes,  opinions,  and  behavior.  Results  of  the  study  will  help  evaluate 
pedagogical  techniques  used  in  educating  present  aiid  future  professionals  about  CAD  as  a  design  tool 
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Methodology 

Sample 

The  sample  was  drawn  from  a  mailing  list  of  associate  and  professional  members  of  the  American 
Society  of  Interior  Designers  (ASID)  and  the  Institute  of  Business  Designers  (IBD)  in  Colorado  and 
Wyoming.  Industry  Foundation  (IF)  members,  educators  and  press  were  excluded  from  the  mailing 
list.  Of  the  349  designers  identified  within  Wyoming  and  Colorado,  94  (27  percent)  returned  a 
postcard  indicating  they  had  never  used  a  CAD  system  and  therefore  were  qualified  to  attend.  Fifty- 
two  practicing  designers  ultimately  participated  in  the  workshops. 

Of  the  participants,  61  percent  were  associate  members  and  39  percent  were  professional  members  of 
ASID  or  IBD.  Seventy-eight  percent  were  female;  55  percent  were  over  35  years  of  age.  All  had 
completed  college  or  comparable  design  school  training  and  57  percent  had  ten  or  fewer  years  of 
design  experience.  Most  (72  percent)  of  the  designers  worked  in  small  design  firms  with  the  same 
percentage  of  firms  having  two  or  fewer  staff  designers.  Sixty  percent  indicated  that  their  work  was 
at  least  50  percent  contract  design. 


In  May  1989,  three  complimentary  workshops,  each  one  day  in  length,  were  presented.  Two  groups, 
19  designers  each,  participated  in  a  hands-on  workshop  during  which  AutoCAD  software  was  used 
by  each  participant.  One  group  of  14  designers  participated  in  a  lecture-style  workshop  during  which 
CAD'S  advantages  and  disadvantages  were  presented  with  demonstrations,  lectures,  slides  and  reading 
material  (N=52),  Designers  were  not  notified  of  workshop  differences.  The  registration  material 
identified  three  different  workshop  dates.  Designers  then  self-selected  the  most  convenient  time  to 
participate  in  a  workshop. 

The  pre-test  was  administered  at  the  workshops  to  all  attending  designers  before  presentation  of  any 
material  on  CAD.  A  post-test  was  sent  one  week  later  and  again  one  year  later  to  assess  changes  in 
the  variables  studied.  Results  presented  in  this  paper  are  based  on  the  post-test  scores  obtained  from 
questionnaires  returned  one  week  later  except  for  the  variable  measuring  behavioral  changes.  The 
same  questions  were  asked  in  the  pre-  and  post-tests,  with  the  exception  of  the  designer's  background 
information  which  was  obtained  only  in  the  pre-test.  The  results  cannot  be  generalized  beyond  this 
sample  but  do  give  some  guides  for  teachers,  researchers,  and  users  of  CAD. 

Variables 

Attitude  Toward  CAD  (see  Table  1)  was  measured  by  asking  respondents  to  check  from  a  list  of  12 
adjectives,  the  three  which  best  described  their  current  attitude  toward  CAD.  Seven  adjectives 
represented  positive  attitudes;  five,  negative  (see  Table  2).  Each  respondent's  score  could  range  from 
+3  (three  positive  adjectives  chosen)  to  -3  (three  negative  adjectives  chosen.  Present  State  of  Mind 
Toward  CAD  was  also  assessed  by  having  respondents  select  three  adjectives.  Respondents  chose 
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those  from  a  list  of  17  (seven  positive,  ten  negative)  which  best  described  their  present  state  of  niind 
toward  CAD  (see  Table  3).  Scoring  again  could  range  from  a  +3  to  -3. 

The  variable  referred  to  on  Table  1  as  Rationale  for  Implementing  CAD  consisted  of  the  sum  of 
participants'  responses  to  14  statements,  each  providing  a  possible  explanation  as  to  why  respondents 
might  not  learn  to  incorporate  CAD  into  their  careers  or  possible  problems  with  its  implementation. 
With  responses  to  each  statement  ranging  on  a  five-point  scale  from  strongly  agree  to  strongly 
disagree,  a  mean  variable  score  was  obtained  with  a  score  of  5  representing  a  strong  rationale  to  learn 
or  implement  CAD  and  a  score  of  1  indicating  a  strong  rationale  not  to  learn  or  implement  CAD. 

The  subject's  Familiarity  with  CAD  systems  was  assessed  by  asking  one  question  to  which  participants 
checked  their  response  as  being  very  familiar  (coded  as  5)  to  not  familiar  (coded  as  1)  with  CAD. 
Using  a  scale  from  five  (favorable)  to  one  (not  favorable),  the  designer's  Feelings  Toward  CAD  were 
sought  by  asking  each  to  respond  to  the  statement,  "In  general,  my  feelings  toward  the  use  of  CAD 
systems  in  the  interior  design  field  is."  Responses  to  the  question,  "If  you  were  to  look  for  a  new  job, 
a  firm  that  used  a  CAD  system  would  be:"  ranged  from  very  important  (5)  to  ncl  important  (1)  and 
measured  the  variable,  CAD's  Imponance  in  New  Job. 

Advantages  of  CAD  Training  asked  respondents  to  cheeky  from  a  list  of  eight  possible  advantages  of 
training  on  a  CAD  system  for  their  career,  all  that  they  perceived  as  possible  advantages.  Scores 
ranged  from  0,  if  none  were  checked,  to  8.  Behavior  Concerning  CAD  Systems  consisted  of  five 
statements  from  which  respondents  chose  the  one  best  representing  tlieir  level  of  laiowledge,  exposure, 
or  use  of  CAD. 


Findings 

Table  1  presents  the  results  of  t-tests  to  determine  if  mean  scores  were  significantly  different  following 
the  workshop  for  hands-on  and  lecture-style  groups.  The  designers'  attitudes  toward  CAD  were 
positive  both  before  and  after  the  workshops  and  were  not  significantly  changed  by  the  type  of 
exposure  they  had  had  to  CAD's  abilities.  The  most  frequently  indicated  adjectives  CTable  2)  used 
to  describe  attitudes  toward  CAD  before  and  after  the  workshops  were  "advantageous,"  "impressive," 
and  ''exciting"  with  over  60  percent  of  the  respondents  checking  these  adjectives  during  both  time 
period.  No  one,  either  pre-  or  post-workshop,  checked  "useless"  co  represent  their  current  attitude. 
The  largest  percentage  point  change  from  pre-  to  post-workshop  was  that  for  the  adjective 
"overwhelming/  with  29  percent  checking  this  adjective  after  the  workshops  compared  with  15 
percent  prior  to  the  workshops. 

The  mean  score  of  the  lecture-style  group  for  the  present  state  of  mind  toward  CAD  variable  increased 
significantly  in  a  positive  direction  from  the  pre-  to  the  post-test  (Table  1).  Only  one  adjective, 
"untrained,"  in  the  pre-test  and  two  adjectives,  "untrained"  and  "favorable,"  in  the  post-test  had  50 
percent  or  more  respondents  checking  ♦hem  (see  Table  3).  However,  the  same  three  adjectives- 
"untrained,"  "eager,"  and  "favorable" -had  the  highest  percentages  of  respondents  checking  them 
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during  both  tests.  The  largest  percentage  point  change  for  this  variable  was  from  38  percent  of  the 
respondents  checking  "favorable"  before  the  workshops  to  60  percent  after  the  workshops. 
Respondents  were  less  likely  to  consider  themselves  "untrained"  (from  68  percent  to  50  percent)  or 
"uneducated"  (from  30  percent  to  13.5  percent)  after  the  workshops;  however,  fewer  stated  that  they 
were  "eager,"  and  more  designers  stated  that  they  were  "hesitant"  after  attending  their  workshop. 

The  rationale  of  the  lecture-style  group  for  learning  or  implementing  CAD  was  significantly  stronger 
after  the  workshop  than  before;  the  mean  score  for  the  hands-on  group  remained  the  same  as  before 
the  workshop  (see  Table  1).  Tv^'ice  as  many  (3 1  percent)  of  the  participants  agreed  with  the  statement, 
"My  career  keeps  me  too  busy  to  learn  how  to  use  CAD,"  after  tlie  workshops  than  before  (15 
percent). 

A  key  finding  was  that  both  the  hands-on  and  tlie  lecture-style  groups  were  significantly  more  familiar 
with  CAD  after  the  workshops  than  before.  Nevertheless,  with  mean  scores  below  the  mid-point  of 
the  familiarity  scale,  workshop  participants  realized  that  they  still  had  more  to  learn  before  being 
completely  familiar  with  CAD. 

Feelings  about  CAD  were  highly  favorable  before  and  after  the  workshop.  Perhaps  individuals  feeling 
negatively  would  not  commit  themselves  to  attending  a  workshop.  The  mean  score  of  the  lecture-style 
group  increased  significantly  after  the  workshop.  Likewise,  the  lecture-style  group's  perception  of 
the  importance  of  CAD  in  a  new  job  increased  significantly  following  the  workshop  whereas  the 
hands-on  group's  perception  did  not. 

As  Table  4  indicates,  chi  square  analysis  showed  a  significant  difference  in  the  behavior  of  workshop 
participants  in  regard  to  CAD  after  the  workshops.  One  year  later,  each  of  the  50  percent  of 
participants  who  initially  lacked  information  about  how  CAD  systems  worked  had  gained  some 
understanding  of  how  CAD  worked  or  had  thought  about  a  system  for  home/work  use.  After  the 
workshops,  23  percent  had  thought  seriously  about  obtaining  a  CAD  system,  had  gotten  one,  or  had 
decided  against  it  compared  with  only  6  percent  prior  to  the  workshop. 

Tlie  opinions  of  participants  about  the  advantages  of  CAD  education  for  enhancing  careers  were 
sought.  As  noted  in  Table  5,  the  possible  advantages  which  were  cited  by  the  highest  percentages  of 
respondents  prior  to  and  after  the  workshops  were:  trained  for  future  needs,  versatility  and  credibility 
within  your  job,  improved  communications  skills  to  v/ork  with  CAD  operators,  and  increased  speed 
at  your  job.  With  the  exception  of  increased  speed  at  your  job,  each  statement  was  cited  by  a  higher 
percentage  of  respondents  as  a  possible  advantage  after  the  workshops  than  before.  However,  the 
difference  was  significant  only  for  job  security.  Over  twice  as  many  (44  percent)  thought  job  security 
was  a  possible  advantage  after  attending  the  workshop  than  prior  to  it  (21  percent). 
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Discussion 


Interior  designers  occasionally  express  a  reluctance  to  adopt  CAD  into  their  design  firms.  Today,  and 
in  the  past,  it  is  common  to  use  lecture-style  pedagogical  techniques  when  educating  professional 
designers  and  educators  about  CAD  systems.  Trade  shows,  professional/education  workshops, 
continuing  education  credits  (CEU's)  and  in-house  training  seminars  are  just  a  few  of  the  areas  where 
lecture-styles  or  "show  and  tell"  demonstrations  typically  take  place.  This  study  was  conducted  with 
the  assumption  tliat  designers,  given  an  opportunity  for  one  day  to  personally  operate  CAD  would 
experience  positive  attitude  and  opinion  modifications  toward  CAD  usage.  The  results  did  not  support 
that  assumption.  However,  the  variable  assessing  the  participating  designers'  behaviors  one  year  later 
indicated  some  positive  changes. 

The  pre-  to  post-test  results  reveal  that  some  participants  were  increasingly  overwhelmed  by  their 
exposure  to  CAD.  An  explanation  may  be  that  many  times  CAD  is  presented  to  designers  by 
professional  computer  personnel  as  a  "magic  show."  Previously  rehearsed  demos  are  typically 
presented  to  the  designer  smoothly  and  perhaps,  unrealistically.  Littie  information  is  offered 
concerning  the  necessary  learning  curve  required  to  develop  design  proficiency  with  a  CAD  system. 
Designers  assume  from  the  CAD  presentations  that  CAD  can  be  learned  quickly  and  that  littie  time 
will  be  sacrificed  from  their  work  day.  Perhaps  their  somewhat  overwhelmed  "state  of  mind"  was  a 
result  of  the  realistic  snapshot,  offered  by  this  workshop,  of  the  time  that  must  be  invested  to 
competenUy  operate  a  CAD  system.  Yet  despite  feeling  increasingly  overwhelmed,  the  designers  still 
believed  CAD  was  "advantageous"  (70  percent),  "impressive"  (73  percent),  and  "exciting"  (64 
percent)  to  learn.  In  addition,  the  pre-post  test  results  revealed  that  the  participants  were  more 
favorable  (38  to  60  percent)  toward  CAD  usage.  Results  revealed  that  the  percentage  of  participants' 
indicating  that  job  security  would  be  one  advantage  of  CAD  education  doubled  from  pre-  to  post-test. 

Significant  differences  between  pre-  and  post-test  for  the  lecture-style  group  were  found  for  five  of 
the  six  variables  presented  in  Table  1.  All  of  the  changes  were  positive.  Only  one  variable  was 
significantiy  changed  for  the  liands-on  group.  It  is  the  opinion  of  the  researchers  that  those  actually 
operating  the  CAD  systems  were  exposed  to  the  learning  curve  required  for  CAD  proficiency. 


Sum.mary 

This  research  sought  to  investigate  whether  attitudes  and  opinions  held  by  practicing  interior  designers 
toward  CAD  differed  depending  on  the  exposure  to  a  hands-on  versus  a  lecture-style  pedagogy.  The 
majority  of  the  sample  was  female,  had  completed  their  college  education  and  was  practicing  at  least 
50  percent  contract  design  in  small  firms.  Significant  differences  in  variable  scores  between  the  pre- 
and  post-tests  occurred  primarily  for  the  lecture-style  group  which  only  saw  demonstrations;  they  did 
not  operate  a  CAD  system.  They  felt  a  stronger  reason  to  learn  or  implement  CAD  after  the 
workshop  and  believed  CAD  to  be  more  important  than  did  the  hands-on  group  in  a  new  job.  One 
day  hands-on  CAD  workshops  for  tiiose  just  learning  to  operate  CAD  were  not  significant  in  changing 
participant's  attitudes  or  opinions.  The  learning  curve  may  be  too  difficult  for  a  one-time  exposure. 
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However,  overall  attitudes  toward  CAD  remained  positive  for  the  hands-on  and  lecture-style  group 
and  both  groups  realized  that  knowing  CAD  would  significantly  increase  their  job  security. 


Future  Areas  of  Research 

One  area  of  further  research  would  be  to  randomly  select  a  sample  of  practicing  designers  who  have 
not  operated  a  CAD  system  and  offer  a  three-  to  five-day  CAD  workshop  to  assess  designers' 
attitudes,  opinions,  and  behavior  modifications  in  relation  to  their  exposure  to  CAD,  Also,  the  current 
study  should  be  replicated  with  a  larger  sample  to  gain  further  insight  into  the  effects  of  presentation 
methodology. 
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Table  1        Mean  Scores  on  Key  Variables  Prior  to  and  Following  a  CAD  Workshop  for 
Respondents  in  Hands  On  (n  =  38)  and  Lecture  Groups  (n  =  14) 


Variables 


1.  Attitude  toward  CAD 

Hands  On 
Lex:ture 

2.  State  of  Mind  toward  CAD 

Hands  On 
L^ture 

3.  Rationale  for  Implementing  CAD 

Hands  On 
Lecture 

4.  Familiarity  with  CAD 

Hands  On 
Lecture 

5.  Feelings  about  CAD 

Hands  On 
Lecture 

6.  Importance  of  CAD  in  New  Job 

Hands  On 
Lecture 


Pre  Post 
Workshop  Workshop  t-Score 


2.1  2.0  .20 
1.7  1.8  -.27 

-0.21  0.13  .89 

-0.29  0.93  -2.72* 

3.3  3.3  .15 

3.2  3.6  -3.70** 

1.7  2.5  -5.92*** 

1.7  2.8  -4.17*** 

4.1  4.2  -1.48 

3.9  4.4  -3.31** 

3.3  3.2  .32 
3.0  3.6  -3.23** 


*  p  <  .05 
**  p  <  .01 
***     p  <  .001 


1  "I  - 
171  i  <  X 


8 


Table  2        Percentages  of  Respondents  Checking  Adjectives  as  Representative  of  Current  Attitude 
Toward  CAD  Systems  (n  =  52) 


Adjective 

Pre 

Post 

Difference  in 
%  Points 

advantageous 

77.4 

69.2 

-8.2 

impressive 

67.9 

73.1 

5.2 

exciting 

60.4 

63.5 

3.1 

amazing 

22.6 

15.4 

-7.2 

foreign 

15.1 

1.9 

-13.2 

overwhelming 

15.1 

28.8 

13.7 

powerful 

11.3 

17.3 

6 

affordable 

7.5 

9.6 

2.1 

unbelievable 

5.7 

5.8 

0.1 

ugly 

1.9 

0 

1.9 

useless 

0 

0 

unapplicable 

0 

5.8 

5.8 

other 

9.4 

3.8 

-5.6 
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Table  3        Percentages  of  Respondents  Checking  Adjectives  as  Representative  of  Present  State  of 
Mind  Toward  CAD  Systems  (n  =  52) 


/vujccuve 

JrOSl 

Difference  in 

/o  JrOiniS 

untrained 

67.9 

50 

-17-9 

eager 

41.5 

26. y 

-14.6 

favorable 

37.7 

59.6 

21.9 

excited 

30.2 

oc 

25 

-5.2 

uneducated 

on  o 
iU.2 

1  o  4r 
13.5 

-16.7 

agreeable 

ZU.O 

19. Z 

-l.o 

intimidated 

15. 1 

7.7 

-7.4 

anxious 

9.0 

-3.0 

hesitant 

0^ 

Z5 

1  1  Q 
11. 0 

conticieni 

1  1  'i 
1 1.  J 

15.4 

A  t 

4. 1 

reiuctant 

/.J 

/.  / 

U.2 

incompetent 

5.7 

1.1 

2.0 

fearful 

3.8 

0 

-3.8 

neutral 

3.8 

1.9 

-1.9 

unaware 

1.9 

1.9 

0 

educated 

0 

5.8 

5.8 

sold 

0 

7,7 

7.7 

other 

0 

1.9 

1.9 
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Table  4        Percentages  of  Respondents  Indicating  a  Particular  Behavior  Concerning  CAD  Systems 
Prior  to  and  Following  a  CAD  Workshop  (n  =  52) 

Question  Pre  Post 

I  have  heard  of  CAD  systems  but  really  do  50  0 
not  know  how  they  work. 

I  have  been  exposed  to  some  information  44.2  76.9 
about  CAD  systems  and  have  some  under- 
standing of  how  it  works. 

I  have  seriously  thought  about  getting  a  5.8  15.4 
CAD  system  for  my  home/work. 

I  have  a  CAD  system  for  my  home/ work.  0  1.9 

I  have  decided  against  a  CAD  system  for  0  5.8 
my  home/work, 

Chi  square     =  36.8 
df  =4 
p  <  .001 
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Table  5        Percentages  of  Respondents  Prior  to  and  Following  a  CAD  Workshop  Perceiving  Each 
Factor  as  a  Possible  Advantage  of  CAD  Training  for  Their  Career  (n  =  52) 


Pre  Post 


Factor  Workshop  Workshop 


1.      Trained  for  future  needs 

81 

85 

2.      Versatility  and  credibility  within 
your  job 

73 

83 

3.      Improved  communications  skills  to 
work  with  CAD  operators 

52 

65 

4.       Increased  speed  at  your  job 

73 

64 

5.      Easier  to  find  a  betier  job  in  the 
market 

44 

60 

6.      Better  salary 

40 

56 

7.       Job  security 

21 

44* 

8.      No  real  advantage  at  this  moment 

4 

12 

Chi  square  =5.29 
df               =  1 
*p               <  .05 

1 75  i 
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INTEGRATING  CONTROVERSY  SKILLS  INTO  COMPUTER  CONFERENCING: 
PARAMETERS  FOR  INSTRUCTIONAL  MODEL  BUILDING 


by 

Dr.  James  L.  Morrison 
University  of  Delaware 

Dr,  Pamela  P.  Morrison 
Goldey-Beacom  College 


ABSTRACT 

An  experiment  in  problem-solving  was  conducted  in  which  one  group 
of  students  used  computer  conferencing  and  another,  class  discussion 
groups  while  attempting  to  solve  a  legal  problem  in  a  course  in 
consumer  law.  Student  success  rates  for  solving  the  assigned  problem 
were  similar.  Based  upon  data  collected,  four  parameters  were 
identified  as  critical  to  integrating  computer  conferencing  as  an 
instructional  strategy.  These  are:  (1)  precise  problem  definition; 
(2)  intensive  preliminary  training;  (3)  pacing  student  inputs/outputs; 
and  (4)  redisti'ibuting  class  time.  The  role  of  the  classroom 
instructor  was  perceived  as  entirely  different  by  members  of  each 
group. 


One  of  the  most  adventurous  types  of  conferencing  becoming  more 
readily  available  to  academicians  is  what  may  be  labeled  as  many-to- 
many  interactive  communication  by  means  of  a  computer.  This  inter- 
active learning  strategy  enables  an  individual  as  part  of  a  large 
group  of  users  to  keyboard  messages  at  personal  computers  located  at 
varying  on-campus  and  off-campus  locations.  These  comments  may  be 
filed  centrally  by  means  of  a  host-mainframe,  thus  enabling  individ- 
uals to  share  ideas  openly  in  a  public  forum.  By  adopting  such  an 
interactive  strategy,  individuals  are  capable  of  entering  into  dialogue 
with  others  while  eliminating  the  barriers  of  time  and  space. 

Alex  Cruz  (1992)  of  American  Airlines  Decision  Technologies 
describes  computer  conferencing  as  a  mechanism  that  enables  individ- 
uals to  communicate  with  one  cr  more  people  through  a  computer  that 
is  connected  to  a  network  of  other  computers.     Edward  Yarrish  (1992) 
describes    computer    conferencing    as    being    somewhat    different  from 
electronic    mail.      Whereas    electronic    mail    customarily    results  in 
private  communication  between  two  individuals ;  computer  conferencing 
adds   the   dimension   of   continual   public   exchanges   of    ideas  through 
asychronous  meetings.      In  this  regard,   Yarrish  categorizes  computer 
conferencing   as   different   time/different   place   technology    in  con- 
trasting it  to  same  time/same  place  learning  environments  typical  of 
the  more  traditional  classroom  discussions  found  in  today's  educational 
institutions. 


If*;  ^ 


Experiment  Conducted 


An  instructional  experiment  was  conducted  for  the  purpose  of 
comparing  student  perspectives  when  utilizing  traditional  classroom 
discussion    or    computer    conferencing    as    an    interactive  learning 
mechanism  when  debating  a  legal  controversy.     The  focus  of  learning 
was  upon  having  undergraduates  apply  prior  learnings  while  thinking 
critically  when  recommending  a  public  policy  related  to  product  safety 
in  the  marketplace. 

During  the  experiment  in  a  consumer  law  course,  48  undergrad- 
uates were  divided  into  two  learning  groups  with  a  control  group  of  24 
students  utilizing  traditional  class  discussions  and  an  experimental 
group  of  24  students  participating  in  a  computer  conference.  Each 
group  was  given  the  identical  charge  of  selecting  and  defending  an 
alternative  for  solving  what  was  presented  as  a  product  liability 
crisis  in  the  marketplace.  Members  of  the  experimental  group  accessed 
personal  computers  using  CONVENE  software  marketed  by  IBM  for  use  on 
the  IBM3090-E  mainframe.  Positions  taken  by  the  experimental  group 
were  forwarded  by  email  to  the  instructor.  For  the  control  group, 
positions  were  presented  in  written  format. 

Data  was  collected  by  means  of  questionnaires  completed  at  the 
conclusion  of  the  2-week  experiment-  Using  a  5-point  rating  scale 
where  a  5  rating  indicated  a  strong  agreement  with  a  statement  provided 
and  1,  strong  disagreement,  students  indicated  their  perceptions  as  to 
specified  aspects  of  the  instructional  strategy  to  which  they  were 
subjected. 

Findings  of  Study 

The  findings  indicate  that  undergraduates  in  both  the  control  and 
experimental  group  perceive  similar  occurrences  in  terms  of  degree  of 
helpfulness,  risk  taking,  maneuvering,  dealing  with  large  number  of 
comments,  determining  significance  of  comments  made,  and  the  degree  of 
difficulty  in  formulating  final  positions  (Table  1).  However,  there  was 
a  significant  difference  in  the  perceptions  of  students  between  the 
control  and  experimental  group  as  to  the  role  undertaken  by  the 
instructor  during  the  actual,  debate. 

Delivery  of  education  through  computer  communication  alters 
the  relationship  between  the  instructor,  student,  and  the  course 
content.  Unlike  traditional  classroom  instruction  in  which  the 
educator  generally  lecis  in  discussion,  prompts  responses,  and  paces 
the  class,  computer  conferencing  is  more  student-centered  with  less 
instructor  involvement.  The  mean  score  of  2.875000  on  the  part  of  the 
experimental  group  indicates  that  students  generally  disagreed  with  the 
statement  that  they  found  it  necessary  to  rely  upon  the  classroom 
instructor  for  leadership  and  direction.  However,  the  mean  score  of 
3.625000  reveals  that  students  in  the  control  group  were  more  in 
agreement  with  the  need  to  seek  and  get  instructor  support. 
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Table  1 

MEAN  SCORES  AND  PROBABILITIES  OF  STUDENT  PERCEPTIONS 
OF  OCCURRENCES  BY  STUDENT  GROUP 


Occurrence  Mean  Mean  T-Score  Prob* 

Exp.  Grp  Control  Grp 


Others  influenced 


Tn\7   +"Vi  1  r;V  "i  nrr 
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•     ^  sj 

.  55 
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J . 
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•J 

9  Q  ft    ft  7 
.  <iS  ^  D  D  D  / 

n 

\j 

7  /1ft 

track  of  POTnments 

3  . 

166667 

JL  \J  \J  kJ  KJ  1 

2 

.  875000 

1 

.  0579 

.  20 

Like  to  take  risk 

expressing  myself 

3  . 

083333 

3 

.125000 

-0 

.1259 

.90 

Used  readings 

identified  by 

others 

3. 

541667 

3 

.416667 

0 

.4248 

.67 

Necessary  to  meet 

outside  of  class 

2. 

666667 

3 

.125000 

-1 

.5235 

*13 

Was  candid  when 

interacting 

3. 

416667 

3 

.333333 

0 

.2851 

.11 

Difficult  to  make 

comments 

3. 

083333 

2 

.625000 

1 

.4728 

.14 

Easy  to  interpret 

sign,  of  comments 

3. 

625000 

4 

.125000 

-1 

.6262 

.11 

Willingness  to 

share  ideas 

3. 

916667 

'> 

.750000 

0 

.7938 

.43 

Relied  upon  Inst. 

for  Guidance 

2. 

875000 

3 

.625000 

-2 

.0180 

.05** 

Computer  Conf/ 

Disc.  Helpful 

3. 

750000 

4.00000 

-1 

.0000 

.32 

**Signif icance  at  .05 

level 

of  confiden 

ce 

The  findings  also  indicated  that  boch  the  control  (mean  score  of 
4.000001  and  experimental  (mean  score  of  3.75000)  groups  found  their 
instructional  strategy  (whether  traditional  class  discussion  or 
computer  conferencing)  to  be  helpful  for  gaining  data  and  support  for 
defining  their  own  position  in  terms  of  the  legal  controversy  under 
debate.  Both  groups  appeared  to  be  candid  with  their  remarks  and  also 
agreed  that  they  did  not  find  it  necessary  to  meet  with  others  outside 
of  class  for  getting  assistance  in  arriving  at  a  position  (Table  1). 

Parameters  for  Instructional  Model  Building 

Several  qualitative  findings  are  worthy  of  note  in  terms  of  con- 
ducting students  through  computer  conferencing  activities.  Since  both 
the  control  and  experimental  groups  generally  succeeded  in  defining  and 
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defending  a  position  that  was  documented  for  credibility,  it  appears 
that  group-oriented  learning  has  considerable  merit.  However,  the  role 
of  the  instructor  may  be  characterized  as  somewhat  different  for  that 
of  computer  conferencing.  In  this  regard,  four  parameters  were 
identified  which  may  be  identified  as  significant  when  structuring  a 
computer  conference  with  the  purpose  of  generating  input  and  exchange 
of  dialogue  among  all  the  participants. 

1.  Structuring  the  Problem-Solving  Situation.  The  problem 
utilized  as  a  focus  of  study  should  be  quite  specific  and  one  in  which 
much  has  been  written  in  the  literature.  By  focusing  upon  an  existing 
contro\rersy ,  students  are  able  to  relate  findings  to  their  own  welfare, 
which  is  important  to  giving  credibility  to  the  process.  Therefore, 
guidelines  as  to  labeling  comments  with  communicative  titles, 
generating  ideas  within  specified  comment  lengths,  and  documenting 
thoughts  through  references  to  current  literature  are  most  critical 
toward  bringing  order  to  what  could  easily  become  chaotic  in  terms  of 
information  overload  (that  is,  rambling  comments  that  never  seem  to 
enr").  Structuring  comments  within  strict  limits  reduces  the  length  of 
ti:  .e  students  are  keyboarding,  reading,  and  recalling  prior  exchanges. 

2.  Student  Preparation.  The  technical  idiosyncrasies  associated 
with  maneuvering  within  a  computer  conference  have  a  tendency  to  mis- 
direct the  student's  attention  to  the  task  at  hand.  Therefore,  it  is 
suggested  that  students  require  "substantial"  prior  practice  in 
accessing,  maneuvering,  and  organizing  thoughts  through  electronic 
dialogue.  Because  of  the  technical  nature  of  computers,  computing 
support  staff  must  be  readily  available  to  assist  users  through  the 
"pitfalls"  of  computing  (especially  editing  sequences).  In  this 
experiment,  students  were  required  to  complete  a  three-hour,  one-week 
training  period  followed  by  a  "practice  controversy"  in  which  each 
participant  entered  into  dialogue  by  initiating  ideas  and  reacting  to 
comments  of  others, 

3.  Pacing  Student  Involvement.  In  order  to  assure  that  students 
did  not  wait  until  the  last  minute  to  participate  in  the  conference, 
specific  deadlines  were  established  by  which  comments  had  to  be 
entered/responded  to  throughout  the  2-week  experiment-  A  minimum 
requirement  of  6  comments  per  week  was  established — two  per  day 
reflecting  traditional  class  meetings — in  order  to  convey  the 
importance  of  helping  others  along  a  continual  paradigm.  In  addition, 
the  length  of  comments  was  restricted  to  no  more  than  2  computer 
screens.  It  should  be  noted  that  a  vast  majority  of  students  met  the 
minimum  criteria  for  participations  over  the  debate  conducted  during 
the  experiment. 

4*  Redistributing  Class  Time.  Undergraduates  participating  in 
the  computer  conference  were  excused  from  traditional  class  Meeting 
over  the  two-week  period.  However,  6  classes  were  held  wxth  the 
control  group  using  a  class  discussion   format  whereas  those   in  the 
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computer  conference  were  required  to  enter  two  comments  per  class  day 
but  within  a  time  frame  of  6  a.m.  to  12  midnight.  Therefore,  each 
group  did  not  interact  with  one  another  and  the  computer  conference  was 
considered  a  substitute  for  class  meetings. 

Significance  of  Experiment 

It  is  worthy  to  note  that  with  computer  conferencing,  the  class- 
room instructor  has  an  additional  responsibility  of  providing  a 
learning  structure  that  enables  students  to  work  out  a  problem,  search 
for  strategies  on  their  own,  and  evaluate  their  solutions.  The  com- 
puter  conference  enables  the  teacher  to  fulfill  the  role  as  an  obser- 
vant with  the  primary  focus  upon  having  student  adapt  individual 
thinking  processes  for  analyzing  and  synthesizing  thoughts  into  a 
logical  conclusion.  The  role  assumed  by  the  instructor  may  be 
described  as  being  alert  for  misinterpretations  on  the  part  of 
participants  and  immediately  correcting  them  to  allow  participants 
to.  proceed  without  unnecessary  delays  or  distractions. 

During  a  computer  conference,  each  participant  finds  himself/ 
herself  in  a  role  of  having  to  participate.  Such  an  occurrence  is  not 
necessarily  true  in  the  traditional  classroom  discussion.  In  face- 
to-face  exchanges,  some  students  may  have  little  opportunity  to 
participate.  Personality  appears  to  become  less  important  in  computer 
conferencing  than  in  traditional  classroom  debating  processes. 

The  findings  of  the  experiment  give  credibility  to  the  argument 
that  students  should  be  rewarded  for  posing  new  alternatives,  for 
changing  their  positions  when  the  evidence  is  substantial,  and  for 
arriving  at  an  informed  decision  reflecting  advanced  problem-solving 
skills.  Therefore,  the  learning  process  itself  becomes  the  focus  of 
attention — and  not  necessarily  any  one  particular  solution  to  a 
controversy. 

If  computer  conferencing  is  adopted,  the  result  is  a  redistri- 
bution of  class  time  in  that  student  involvement  moves  from  that  of 
primarily  interacting  in  a  controlled  classroom  environment  to  that 
of  a  campus  perspective  whereby  library  resources  and  computing 
capabilities  are  integrated  into  a  learning  network.  Therefore,  in 
this  respect,  it  is  the  promoting  and  managing  of  controversy 
constructively  that  requires  students  to  think  out  their  ideas 
thoroughly,  to  gather  additional  information  when  circumstances 
suggest  a  need,  and  to  sequence  prior  learnings  into  a  logical  order 
for  final  presentation.  In  this  regard,  computer  conferencing  has 
considerable  merit  as  a  learning  strategy. 
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Teaching  with  HyperCard:  An  Innovative  Case  Study 

Bryan  R.  Cole,  Texas  A&M  University 

This  presentation  outlined  the  conceptualization,  development  and  implementation  of  the 
configuration  of  a  course,  "Higher  Education  Law"  utilizing  HyperCard  programming  on  the 
Macintosh  computer.  The  presentation  sought  (1)  to  expand  participant  understanding  and 
thinking  about  the  improvement  of  teaching  and  learning  through  the  utilization  of  HyperCard 
as  an  instructional  delivery  system,  (2)  to  share  with  session  participants  lessons  learned  from 
the  development  of  an  idea  into  a  computer-based  instructional  delivery  system,  and  (3)  to 
provide  session  participants  with  an  opportunity  to  interact  directly  on  issues  impacting  upon  the 
utilization  of  HyperCard  in  an  on-line  instructional  environment.  Emphasizing  pedagogical 
rather  than  technical  considerations,  the  presentation  outlined  the  course  methodology  utilizing 
a  HyperCard  database  of  over  1300  case  briefs  which  serve  as  an  on-line  retrieval  system  that 
effectively  represents  the  case  law  of  higher  education  administration.  Cases  can  be  retrieved 
from  the  database  using  three  different  indexes  (I)  case  name,  (2)  subject,  and  (3)  topic. 
Accordingly,  the  database  can  be  utilized  in  conjunction  with  a  Macintosh  computer,  overhead 
projector,  and  an  overhead  projector  LCD  projection  panel  to  provide  real  time  access  during 
class  instruction  to  this  complex  and  comprehensive  database.  This  methodology  has  now  been 
utilized  in  three  graduate  courses.  The  presentation,  therefore,  also  included  evaluation  and 
recommendations  as  a  result  of  the  pilot  period  and  offered  suggestions  to  others  who  may  want 
to  utilize  HyperCard  to  direct  support  of  course  offerings.  The  session  included  a  demonstration 
of  the  database  and  how  it  is  used  in  the  course. 


Simulating  the  Reasoning  Process  with  Causal  Modeling  Techniques 

Joanna  C.  Dunlap  and  David  H.  Jonassen,  University  of  Colorado 

Instructional  computer-based  simulations,  through  the  presentation  of  realistic  scenarios  in 
situated  settings,  attempt  to  provide  learners  with  environments  in  which  they  can  safely  practice 
using  their  skills  and  knowledge  to  solve  problems  and  make  decisions.  One  of  the  problems 
with  using  a  simulation  as  an  instructional  strategy  is  that  it  can  often  be  difficult  to  realistically 
represent  the  complexity  that  is  part  of  actual  "real  world"  decision  making  and  problem  solving; 
decision-making  and  problem  solving  often  happen  under  a  veil  of  uncertainty  in  which  there  are 
many  possible  reasons  for  and  causes  of  each  event/problem  and  just  as  many  possible  solutions 
for,  answers  to,  or  effects  of  each  event/problem.  Using  causal  modeling  techniques  (statistical 
techniques  that  facilitate  the  speculation  of  causal  linkages  among  correlations),  instructional 
designers  are  able  to  provide  simulated  learning  environments  that  enable  learners  to  practice 
being  problem  solvers  and  decision  makers  under  uncertainty. 

Causal  modeling  theory  was  used  as  a  basis  for  structuring  a  learning  environment  for  medical 
students  in  the  ai'ea  of  transfusion  medicine  problem  solving  and  decision  making  that  simulates 
the  reasoning  process  involved  in  addressing  complex  clinical  situations. 

This  presentation  demonstrated  how  instructional  simulations  can  be  structured  using  causal 
modeling  techniques. 
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The  Media  Mix 

Stephen  W.  Harmon,  The  University  of  Houston  -  Clear  Lake; 
Thomas  C  Reeves,  The  University  of  Georgia 

Everybody's  talking  about  multimedia,  but  nobody  seems  to  be  doing  anything  about  it.  This 
session  presents  a  panel  of  experts  in  interactive  multimedia,  discussing  some  of  the  "why's" 
"why  nets"  and  "how-comes"  of  multimedia  project  development.  Speaking  from  experience 
garnered  in  business  and  industry,  the  military,  and  university  settings,  the  panelists  will  present 
their  views  of  managing  multimedia  projects,  discuss  special  problems  in  creating  multimedia 
applications  and  will  answer  questions  from  the  floor  (and  from  members  of  the  audience!) 

A  wide  variety  of  project  management  concepts  have  gained  popularity  recently, mm  and  many 
of  them  have  made  their  way  into  instructional  settings  as  well.  Panelists  will  discuss  Total 
Quality  Management  (TQM),  rapid  prototyping,  traditional  ISD,  and  other  approaches  to 
creating  effective  multimedia  projects.  Panelists  will  discuss  case  studies  of  projects  already 
completed  and  currently  underway. 


An  Examination  of  Novice  Linking  in  Hypermedia  Environments 

Steve  V/,  Harmon,  University  of  Houston  -  Clear  Lake 

The  goal  of  this  study  was  to  evaluate  how  novice  users  of  hypermedia  systems  carried  out  the 
task  of  linking  nodes  of  information  in  a  relatively  unfamiliar  content  area.  The  hypermedia 
system  was  designed  to  present  the  nodes  as  separate  topics  with  no  pre-imposed  structure  and 
no  suggestion  of  relationships  between  topics.  Data  was  collected  through  observation, 
interview,  link  capture,  pre-treatment  survey,  post-treatment  questionnaire  and  a  post-treatment 
individual  differences  measurement  test.  Eight  different  types  of  links  were  identified  by  the 
subjects. 

The  first  two  of  these  are  similar  and  opposite  links,  used  when  nodes  were  either  alike  or 
opposed.  The  third  link  type  is  causal,  resulting  when  one  node  was  the  result  of  another  node. 
The  fourth  link  type  is  sequential,  identified  when  one  node  occurred  chronologically  before  or 
after  another.  The  fifth  link  type  is  associative,  describing  links  where  one  node  expliciUy 
mentioned  another.  The  sixth  link  type  is  exemplary^  used  for  links  between  nodes  that 
illustrated  each  other  exemplary.  The  seventh  link  type  is  componential,  used  when  one  node 
was  a  part  of  another  one.  The  eighth  link  type  is  accidental,  used  when  subjects  made  a 
mistake  when  linking  two  nodes.  This  session  fully  describes  the  study,  and  discusses  its  results 
and  implications. 
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Cognitive  Flexibility,  Epistemic  Beliefs,  and  Hypertext  Learning  Environments: 
Research  into  the  Transfer  of  Complex  Knowledge 

Michael  J.  Jacobson,  The  University  of  Illinois  at  Urbana-Charnpaign 

The  utilization  of  hypertext  and  hypermedia  systems  for  instructional  purposes  continues  to 
receive  considerable  attention.  Although  research  is  starting  to  document  the  instructional 
efficacy  of  learning  environments  based  on  hypertextual  technologies,  a  number  of  psychological 
and  educational  dimensions  associated  with  the  instructional  uses  of  these  systems  remain  to  be 
explored.  In  an  earlier  hypertext  study,  Jacobson  and  Spiro  (1991)  found  significant  knowledge 
transfer  outcomes  associated  with  hypertext  design  elements  that  were  explicitly  based  on  a 
recent  theory  of  learning,  cognitive  flexibility  theory.  This  presentation  discusses  a  follow-up 
study  examining  different  degrees  of  learner  control  over  a  cognitive  flexibility  hypertext 
procedure  that  provides  conceptual  and  case  "criss-crossings"  of  hypertext  nodes  to  demonstrate 
structural  knowledge  interrelationships.  It  is  anticipated  that  epistemic  beliefs  about  the  nature 
of  learning  and  metacognitive  factors  related  to  the  learner's  control  of  the  hypertext  exploration 
paths  will  interact  with  the  ability  of  the  students  to  learn  and  to  transfer  their  knowledge  after 
using  the  experimental  hypertext  materials.  The  implications  of  this  study  will  be  discussed  in 
terms  of  design  issues  for  hypertext  learning  environments  and  cognitive  factors  associated  with 
knowledge  acquisition  and  transfer. 


Conceptual  and  Methodological  Problems  in  Hypennedia  Research 

David  H.  Jonassen,  University  of  Colorado 

In  order  to  yield  coherent  research  that  can  guide  the  design  of  hypermedia  systems  for  learning,  a 
systematic  research  agenda  must  be  impleme;.ted  to  investigate  the  following  issues  (these  are  the  issues, 
I  claim,  that  most  affect  learning  from  hypermedia): 

Appropriateness  of  hypermedia  applications  and  the  implications  for  meaningful  learning  resulting  from 
them 

Information  retrieval  hypertexts 
Complex,  constructivistic  learning  environments 
Collaborative  knowledge  construction 
Assumptions  of  hypermedia  for  learning 
Learner  control 

Meaningful  purpose  or  goal  for  learning 
Design  and  procedural  issues 

User  functionality  in  the  interface 

Intellectual  functionality  in  the  structure  and  facilities 

User  literacy 

Relationship  of  functionality  to  learning  goal 
Evaluating  outcomes  from  hypermedia  use 

Process  outcomes  from  hypermedia  use 
Learning  outcomes 

Relevance 

Depth  of  processing 

Relationship  to  process  outcomes 
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Collaborative  Knowledge  Construction  in  an  Argument-Based  Hypertext 

David  H.  Jonassen,  University  of  Colorado 

Computer  supported  intentional  learning  environments  (CSILEs)  provide  students  with  a  forum 
for  subject  matter/content  area  exploration  based  upon  individual,  self-determined  needs  and 
interests  (Scardmalia  et  al,  1989;  Scardmalia  &  Bereiter,  1991).  This  presentation  describes  a 
hypertext  shell  to  facilitate  argumentation  of  issues  in  an  advanced  knowledge  seminar. 
Individuals  were  required  to  attach  issue  links  to  these  content  nodes.  To  each  issue  node, 
personal  position  and  alternative  model  links  can  be  automatically  created  to  reference  nodes 
described  by  those  links.  To  each  of  these  position  nodes,  argument  nodes  and  links  can  be 
created-  The  outcomes  of  imposing  this  type  of  formal  argument  structure  onto  the  collaborative 
construction  of  a  class  text  include  more  elaborate  class  discussions  and  better  argued  class 
papers. 


The  Application  of  Research-Based  Instructional  Design  Principles  in  [developing 
a  Hypemiedia-Assisted-Instructionai  Courseware  for  Second  Language  Learning 

Min  Liu,  West  Virginia  University 

The  nonlinear,  associative,  flexible  and  efficient  characteristics  of  hypermedia  have  excited  many 
educators.  Due  to  these  obvious  advantages,  hypermedia  is  becoming  a  popular  tool  for 
courseware  developments  However,  hypermedia  has  its  limitaUons  such  as  disorientation, 
cognitive  overload  and  "too  much"  learner  control.  To  develop  a  quality  hypermedia-based 
courseware,  efforts  must  be  made  to  take  care  of  these  inherent  limitations.  Research  has  shown 
that  a  knowledge  of  instructional  design  theories  and  principles  is  critical  for  designing  effective 
CAL 

This  presentation  intends  to  discuss  ten  research -based  instructional  design  principles  that  are 
most  relevant  to  design  courseware  using  hypermedia  and  how  these  principles  can  be  used  to 
maximize  the  potential  of  hypermedia  as  well  as  to  overcome  its  limitations.  It  will  also  discuss 
the  author's  experience  of  implementing  these  principles  in  developing  a  semantic-network-based 
hypermedia  courseware  for  second  language  learning. 


Information  Seeking  -  A  Fundamental  Learning  Activity 

Carol  B.  MacKnight,  University  of  Massachusetts 

Searching  for  information  in  documents  is  an  activity  that  is  carried  out  by  all  of  us,  first  as 
students  and  then  as  employees.  We  search  dictionaries,  encyclopedias,  textbooks,  manuals,  and 
the  like  for  a  solution  to  a  problem.  A  first  step  in  any  learning  process  is  satisfying  our 
informational  need,  that  is,  seeking  information  that  answers  a  question,  solves  a  problem,  or 
helps  us  to  comprehend.  The  importance  of  :his  task  is  highlighted  by  Kirsch  and  Guthrie 
(1984)  who  found  that  adults  in  the  workplace  spend  n^tore  time  on  reading  to  locate  information 
than  reading  for  any  other  purpose. 
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How  students  search  electronic  documents  and  how  effective  they  are  in  this  process  is  the 
subject  of  our  study.  We  wanted  to  know  which  navigation  tools  student  would  choose  for  a 
particular  task,  how  effectively  and  efficiently  they  used  the  navigational/structural  aids  to  locate 
the  information,  what  types  of  errors  were  made,  and  where  parts  of  the  hypertext  system  need 
improvement  or  redesign. 

After  a  brief  introduction  to  the  system,  sixteen  undergraduate  students  searched  a  hypertext 
version  of  the  Publication  Manual  of  the  American  Psychological  Association  (APA  Manual), 
which  consisted  of  seven  chapters  and  over  1,000  cards,  for  answers  to  six  questions. 

Students  were  encouraged  to  work  as  quickly  as  possible  and  asked  to  write  their  answers  on 
paper.  The  total  time  spent  answering  a  question  was  recorded  from  the  nioment  the  students 
pressed  a  navigation  button  signifying  the  beginning  of  the  search  process  and  ended  with  student 
signalling  the  beginning  of  the  writing  process.  Student's  choices  of  navigation  tools  and  their 
path  through  the  material  were  computer  recorded,  A  comparison  was  made  between  the  best 
path  and  the  path  chosen  by  the  student.  Other  comparisons  and  analyses  were  made  with 
tentative  results  showing  that  there  were  no  significant  differences  in  time  required  to  search  an 
electronic  document  and  time  spent  searching  a  paper  document. 

A  Course  Design  for  Going  Beyond  HyperTalk 

Rose  M.  Marra,  AT&T  Bell  Labs;  R.  Scott  Grabinger,  University  of  Colorado 

Instructional  Technology  curricula  often  stop  at  the  Hyper  Talk  level.  Although  this  provides 
much  functionality  and  satisfies  many  learners,  there  is  a  class  of  problems  ihe  HyperTalk 
language  cannot  address. 

For  example,  gaining  direct  access  to  certain  parts  of  the  hardware  is  not  possible  with 
HyperTalk.  In  addition,  other  operations  such  as  sorting  large  stacks  and  doing  complex  linking 
operations  are  cumbersome  to  perform  within  HyperTalk. 

If  students  wish  to  be  completely  proficient  in  using  HyperCard/HyperTalk  to  create  educational 
solutions  of  any  scale,  a  way  beyond  these  limitations  must  be  available.  As  HyperTalk  does 
not  offer  an  internal  solution,  one  must  look  outside  of  HyperTalk. 

This  session  will  present  a  course  design  for  teaching  students  to  use  "C,  or  another  high-level 
programming  language  such  as  Pascal,  to  write  commands  external  to,  but  accessible  from 
within  HyperTalk  scripts.  HyperTalk  provides  a  means  to  access  these  externally  written 
commands  and  functions;  or  as  HyperTalk  refers  to  them,  XCMDs  and  XFNCTs.  Implementing 
these  XCMDs  and  XFNCTs  in  a  high-level  language  provides  the  functionality  not  available  in 
HyperTalk. 
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Wildlife  Wonders:  Validating  a  Prototype  for  Multidisciplinary 
Education  for  Middle  School 

Gayle      Munson  and  Eric  E.  Smith,  University  of  Northern  Colorado 

The  goal  of  Wildlife  Wonders  is  to  teach  students  to  creatively  solve  problems  and  to  appreciate 
the  world  around  them.  This  is  accomplished  by  putting  creative  tools  in  the  hands  of  students 
while  presenting  information  about  the  natural  world.  Problems  to  be  solved  are  presented  by 
the  teacher  or  generated  by  the  students.  The  problem  might  be  "create  a  fable  about  a  whale" 
or  "do  a  report  comparing  whaling  in  the  last  century  with  today."  The  student  then  has  a 
resource  to  begin  exploring  the  issue.  With  guidance  from  the  teacher,  the  student  learns  to 
solve  the  problem  and  create  and  present  the  solution.  To  be  successful,  the  student  must 
combine  problem  solving  with  research  with  content  with  writing  and  other  disciplines.  The 
process  of  combining  the  approaches  leads  to  more  excitement  and  the  development  of 
"multidisciplinary  eyes"  needed  to  reach  solutions  to  real-worid  problems. 

This  presentation  disciisses  the  design  of  the  material  and  the  results  of  a  validation  study 
conducted  in  the  Laboratory  School  at  the  University  of  Northern  Colorado.  The  Wildlife 
Wonders  project  is  the  first  piece  of  the  Taliesin  Project  vision  to  be  completed  and  validated. 
The  Taliesin  Project  (Smith  &  Westhoff,  1992)  is  aimed  at  creating  a  multidisciplinary 
learning/teaching  environment  and  attitude. 


Hypermedia  Development  Issues:  Validated  Guidelines  and  Research  Needs 

Gayle  Af.  Munson,  Tom  Lightner,  and  Eric  E,  Smith,  University  of  Northern  Colorado 

Practitioners  involved  in  developing  hypermedia  education  environments  are  currently  producing 
their  systems  in  a  standardless  world,  producing  more  uncertainty  for  students,  end  users,  and 
novice  developers.  Different  developers  may  use  different  interface  standards  compounding  the 
confusion.  There  are  contrasting  opinions  about  which  navigational  scheme  to  use  when,  such 
as  icons,  menus,  or  maps,  to  best  mediate  content  learning  in  specific  situations,  leaving  the  user 
without  familiar  direction.  Screens  are  frequently  designed  without  convention  as  to  information 
density  and  placement  of  text,  graphics,  and  controls,  resulting  in  student  focus  on  interface 
rather  than  content.  Designers  may  be  unfamiliar  with  different  methods  for  structuring 
information.  They  may  devise  a  product  using  a  hierarchical  structure  when  node-linked  is  more 
appropriate.  Or  they  might  use  either  structure  type  without  adequately  preparing  the  user  to 
follow  the  information. 

A  comprehensive  set  of  guidelines  for  developers,  for  students,  and  other  users  is  needed. 
Currently  there  are  many  rex:ommendations  in  areas  such  as  information  structures,  screen 
design,  navigational  schemes,  and  the  use  of  icons,  but  little  substantiated  evidence  collected 
regarding  those  recommendations.  This  presentation  examines  each  of  these  issues  and  the 
relevant  empirical  research  to  date  surrcunding  each  issue.  The  presentation  also  highlights 
areas  deserving  more  study. 
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Effectiveness  of  Interactive  Video  on  Tennis  Skill  Analysis 

Gayle  M.  Munson  and  Eric  E.  Smith,  University  of  Northern  Colorado 

The  purposes  of  this  study  were:  (a)  to  develop  a  computer-based  interactive  video  instruction 
system  for  tennis  skill  analysis  for  preservice  physical  education  teachers,  (b)  to  determine  the 
effectiveness  of  the  computer-based  interactive  video  instruction  system  on  preservice  physical 
eaucation  teachers'  psychomotor  skill  analysis  ability,  on  preservice  physical  education  teachers' 
cognitive  skill  analysis  ability,  and  on  preservice  physical  education  teachers'  motor  skill 
performance.  Twenty-one  undergraduate  physical  education  majors  enrolled  in  an  analysis  of 
movement  course  in  tennis  at  the  University  of  Northern  Colorado  participated.  The  1 1  subjects 
in  the  experimental  group  received  instruction  on  the  tennis  skill  analysis  using  the  interactive 
videodisc.  The  10  subjects  in  the  control  group  received  conventional  instruction  from  the 
investigator. 

The  pre-  and  post-test  control  group  design  was  implemented  to  determine  effects  of  the 
treatment  from  pre-  to  post-test.  The  data  were  collected  using  three  types  of  instrumentation: 
(a)  written  cognitive  skill  analysis  test,  (b)  tennis  skill  performance  test,  and  (c)  psychomotor 
skill  analysis  test  using  the  videodisc.  An  analysis  of  covariance  (ANCOVA)  determined  no 
significant  difference  between  the  groups'  cognitive  knowledge  of  tennis  skill  analysis, 
psychomotor  skill  analysis  competency,  or  tennis  skill  performance.  An  analysis  of  variance 
(ANOVA)  determined  significant  improvement  form  pre-  to  post-test  for  both  groups  on  all  three 
dependent  measures.  These  results  suggest  that  the  use  of  interactive  video-based  instruction  for 
psychomotor  skill  analysis  has  great  potential  for  physical  education  teacher  training. 


Evaluating  Hypermedia  Environments:  Insights  from  the  Perseus  Project 

Delia  Neuman,  University  of  Maryland 

Perseus  is  a  multimedia,  interactive  library  of  fifth-century  Greek  history,  literature,  art,  architecture, 
geography,  and  culture.  Created  by  a  team  of  classicists  at  various  universities  and  headquartered  at 
Harvard,  Perseus  has  been  under  development  for  approximately  five  years,  with  major  support  from  the 
Annenberg/CPB  Project  and  additional  funding  from  Apple  Computer  and  the  National  Endowment  for 
the  Arts. 

The  Perseus  Evaluation  Team  of  Gary  Marchionini  and  Delia  Neuman  (University  of  Maryland)  and 
Kenny  Morrell  (St.  Olat  College)  have  used  both  qualitative  and  quantitative  methods  to  conduct  an 
ongoing  formative  evaluation  of  the  emerging  Perseus  materials.  Observations,  interviews,  and  various 
kinds  of  document  analysis  have  been  conducted  at  the  project's  primary  evaluation  sites  —  Bowdoin 
College,  Harvard  University,  University  of  Maryland,  and  St.  Olaf  College  -  while  questionnaires  have 
been  the  primary  data  collection  strategy  at  approximately  25  beta  sites. 

Results  to  date  have  indicated  a  mechanical  advantage  for  Perseus,  the  existence  of  some  qualitative 
differences  in  students'  learning  with  Perseus,  a  variety  of  insights  about  system  performance,  a  need  for 
both  physical  and  conceptual  infrastructures  to  enhance  the  value  of  Perseus,  and  a  concern  for  managing 
the  cognitive  overloads  students  experience  with  the  system » 
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Building  Blocks  -  A  Literacy  Program 

Laurie  Rosenthal,  Teachers  College/Columbia  University 

Intergenerational  literacy  centers  are  places  where  computers  can  profitably  be  used  to  improve 
learning  by  at-risk  populations.  Consequently,  there  is  a  need  for  software  specifically  designed 
to  capitalize  on  the  uniqi»e  learning  situations  present  in  these  settings. 

Building  Blocks,  a  HyperCard  based  hypermedia  program,  meets  this  need  by  integrating 
educational  ideas  with  cognitivist  theory  to  improve  users'  skills,  as  well  as  increase  their 
knowledge.  Intergenerational  dyads,  consisting  of  a  preschooler  and  an  adult,  work  on  activities 
that  promote  cooperative  learning,  while  providing  children  v/ith  positive  adult  role  models,  and 
adults  with  experiences  that  develop  a  sense  of  empowerment. 

Building  Blocks  consists  of  several  stacks  including  a  storybook,  and  an  alphabet.  Constructivist 
ideas,  branching,  rich  graphics,  animation,  sound,  and  a  whole  language  approach  to 
reading/writing  are  used  in  structuring  activities,  and  to  make  information  interesting  and 
meaningful.  A  direct  object  manipulation  user  interface  makes  the  program  engaging  and  highly 
interactive,  and  serves  to  stimulate  discussion  and  question  asking.  Preliminary  tests  show 
Building  Blocks  is  easy  to  use,  holds  user  interest,  and  increases  user  knowledge. 


Developmental  Formative  Research  on  Interactive  Materials 

Erik  Strommen,  Children's  Television  Workshop;  Barbara  Flagg,  Multimedia  Research; 
Patricia  R.  Garretson,  IBM  Corporation;  Henry  Olds,  Bolt,  Beranek  &  Newman 

Developmental  formative  research  is  the  evaluation  of  education  materials,  especially  technology- 
based  materials,  for  children  conducted  during  the  development  process.  The  goaJ  is  to  ensure 
that  the  design  and  content  of  educational  technology  applications  are  appropriate  to  the  intended 
learner  population.  The  present  panel  will  report  on  a  wide  variety  of  formative  research 
projects  conducted  to  address  different  aspects  of  educational  technology  use  in  different  settings, 
and  an  open  discussion  of  issues  in  formative  evaluation  will  follow. 
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A  Strategy  for  Updating  Large  Hypertext  Syste:as 

John  M.  Jackson 
Office  of  Medical  Education 
University  of  Virginia 

Abstract 

While  many  sophisticated  authoring  systems  exist  for  creation  of  hypertext 
programs,  no  authoring  tools  are  available  that  reliably  and  automatically  revise 
manually  created  hypertext  links  when  program  contents  are  revised.  For  large 
hypertext  programs,  this  can  mean  manually  re-creating  thousands  of  hypertext  links 
each  time  the  contents  are  updated.  This  article  presents  a  technique  for  automatically 
reestablishing  any  number  of  hypertext  links  after  a  program's  contents  have  been 
revised.  Tlie  technique  does  not  require  any  hidden  codes  and  authors  may  use  word 
processors  to  revise  program  contents.  The  technique  may  be  used  with  many 
hypertext  authoring  systems. 


Introduction 

Although  the  concepts  and  theory  of  hypertext  have  been  around  for  several 
decades  (Bush,  1945;  Nelson,  1987),  commercial  authoring  systems  did  not  become 
available  until  the  mid  I980's.  Since  then,  software  tools  have  proliferated,  first  on  mini 
and  mainframes,  then  spreading  onto  microcomputer  platforms.  Today  hypertext 
designers  have  many  tools  to  choose  from  according  to  ihe  needs  of  their  projects.  As 
hypertext  software  has  evolved,  so  has  the  research  guiding  the  structure  and  interface 
design  of  hypertext  documents  (Conklin,  1987;  Shneiderman,  Kreitzber,  &  Berk,  1991). 
While  there  is  still  much  to  be  learned  about  the  appropriate  structure  of  hypertext,  w:e 
are  developing  a  good  understanding  of  how  to  build  documents  that  are  easy  to  use 
and  navigate  through. 

Hypertext  research  has  also  expanded  into  the  automatic  conversion  of  documents 
into  hypertext  form.  Commercial  packages  take  several  approaches  to  build  links 
automatically.  Some  use  an  index  or  key  word  list  and  build  links  to  all  occurrences  of 
those  terms  (Riner,  1991).  Others  use  lexical,  statistical,  and  syntactical  analysis  as  well 
as  underlying  semantics  to  intelligently  build  links  between  related  concepts  (Rearick, 
1991;  Salton,  1989).  These  approaches  look  at  the  meanings  of  words,  how  words  are 
used,  and  how  often  they  are  used  to  analyze  the  content.  Hypertext  links  are  then 
build  automatically  to  tie  together  related  topics  and  concepts. 

Another  class  of  automatic  conversion  tools  uses  the  Standard  Generalized  Markup 
Language  (SGML)  to  automatically  construct  hypertext  links.  SGML  is  a  meta-language 
that  has  grown  out  of  the  publishing  industry  that  defines  the  structure,  relationships, 
and  hierarchy  of  a  document  through  a  tagging  system.  SGML  is  commonly  used  in 
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mini  and  mainframe  publishing  systems,  but  encouraged  by  ISO  8879  that  defines  and 
standardizes  SGML's  characteristics,  SGML  is  now  being  incorporated  into  some 
microcomputer  software  (Goldfarb,  1986).  While  SGML  is  primarily  intended  to 
simplify  the  printed  publishing  process,  hypertext  tools  can  use  embedded  SGML  tags 
to  analyze  document  structure  and  quickly  build  links  without  resorting  to  complex 
and  sometimes  unreliable  syntactical  analysis  (Staff,  1991).  Unfortunately  few 
microcomputer  based  word  processors  can  create  SGML  tagged  text,  though  a  few 
utilities  are  available  to  convert  word  processed  files  into  SGML  format. 

Automatic  text-to-hypertext  conversion  software  is  useful  for  developing  reference 
systems,  technical  documentation,  or  other  hypertext  applications  that  allow  the  user  to 
freely  navigate  through  material.  Automatic  conversion  to  hypertext  is  usually  not 
applicable  to  computer  based  instruction  (CBI)  that  guides  the  learner  through  a 
structured  learning  experience.  Typically  the  CBI  author  would  not  be  satisfied  with  a 
hypertext  document  that  simply  cross  referenced  every  occurrence  of  a  word.  As  a 
result  most  hypertext  CBI  systems  use  links  that  are  built  manually  by  the  author. 

Since  hypertext  authoring  systems  have  been  on  the  market  for  several  years,  there 
are  now  thousands  of  large  hypertext  systems  in  place.  They  are  used  in  on-line 
reference  systems,  just-in-time  training  modules,  technical  documentation,  and  many 
educational  packages  including  computer  based  instruction  modules.  The  proliferation 
of  hypertext  systems,  and  the  unavoidable  obsolescence  of  the  information  they  contain, 
has  given  rise  to  the  problem  of  hypertext  system  maintenance. 

Problems  of  Hypertext  Revision 

There  are  many  problems  associated  with  hypertext  maintenance,  depending  on 
how  the  system  was  created.  If  an  automated  text-to-hypertext  conversion  tool  was 
used  the  process  is  fairly  straightforward,  especially  if  the  authoring  tool  maintains  a 
separate  list  of  node  keywords  that  can  be  used  with  newly  revised  documents.  CBI 
authors  unfortunately  cannot  apply  this  technique  since  they  typically  build  most  of 
their  links  manually.  If  an  author  has  used  an  overlay  button  based  tool,  such  as 
HyperCard,  with  invisible  buttons  placed  over  text,  each  time  the  text  is  edited  all  of  the 
buttons  and  related  links  are  pushed  out  of  registration.  To  further  contribute  to  the 
problem,  most  hypertext  authoring  tools  do  not  offer  the  full  functionality  of  dedicated 
word  processors,  and  do  not  include  spelling  and  grammar  checkers,  thesauruses,  or 
style  sheets.  This  makes  it  very  inconvenient,  if  not  impossible,  to  use  hypertext 
authoring  systems  to  edit  documents.  If  a  hypertext  authoring  system  uses  hidden 
embedded  codes  to  define  links,  links  can  also  be  damaged  during  the  text  editing 
process.  All  of  these  factors  contribute  to  making  hypertext  revision  a  very  labor 
intensive  and  frustrating  process,  especially  for  a  subject  matter  expert  or  author  who 
does  not  work  with  these  tools  on  a  daily  basis.  Ideally,  the  hypertext  authoring  system 
should  let  the  subject  matter  expert  use  the  word  processor  of  his  choice  to  compose  and 
revise  his  documents.  The  ideal  system  would  automate  repetitive  authoring  tasks, 
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including  the  link  rebuilding  process  that  takes  place  after  any  major  document 
revision.  The  project  described  below  was  developed  to  meet  these  goals  for  one 
specific  hypertext  environment.  The  strategies  developed  in  this  project  may  be  applied 
to  many  other  types  of  hypertext  systems. 

Project  Description 

In  1990  Dr.  Brian  Duling,  a  physiologist  at  the  University  of  Virginia  Medical 
School,  proposed  a  large  hypc^aext  system  that  would  provide  medical  students  with  a 
set  of  computerized  practice  questions  for  their  review  before  examinations.  Each 
question  would  have  feedback  for  each  possible  response  and  be  cross  referenced  to  an 
extensive  set  of  hypertext  course  handouts  that  would  provide  more  detailed 
explanations.  The  students  and  faculty  would  also  be  able  to  search  through  questions 
or  handouts  for  specific  terms  within  a  given  course  section  or  across  several  courses. 
Formatted  text  and  graphics  for  both  questions  and  handouts  were  to  be  imported  from 
existing  word  processor  files.  The  documents  were  heavily  formatted  and  contained 
many  illustrations,  scientific  formulae,  and  Greek  characters.  Dr.  Duling  sought  to 
design  a  system  that  encompassed  the  entire  first  two  years  of  the  medical  curriculum  at 
the  University  of  Virginia,  including  several  thousand  questions  and  many  thousands 
of  pages  of  course  handouts.  As  many  as  100  faculty  authors  would  be  involved  in  the 
project  at  its  maturity.  Key  to  the  project  was  that  all  course  handouts  would  be  revised 
annually  along  with  all  the  related  hypertext  links  to  the  practice  questions.  Questions 
would  also  be  revised,  but  on  a  less  regular  basis.  Basic  assumptions  were  that  the 
system  must  be  extremely  easy  to  use  by  the  students  and  faculty  authors,  and  that  the 
faculty  must  be  able  to  use  their  word  processors  to  write  and  revise  all  documents. 

Dr.  Duling's  laboratory  built  the  first  prototypes  using  HyperCard®  to  define  the 
types  of  interactions  and  interfaces  necessary  for  the  system.  Using  the  version  of 
HyperCard®  available  at  the  time,  it  was  necessary  to  rely  upon  codes  embedded  in  the 
course  handouts  to  maintain  hypertext  links  during  the  revision  process.  HyperCard® 
also  could  not  easily  import  the  formatted  text  and  scientific  equations  essential  to  the 
project.  The  HyperCard®  prototype  helped  define  the  needs  of  the  proposed  system, 
but  it  was  concluded  that  it  did  not  include  many  features  essential  for  the  success  of 
the  project. 

After  an  extensive  review  of  hypertext  tools  available  at  the  time.  Dr.  Duling  and 
the  author  developed  a  second  prototype  using  Guide  3.0®  running  under  Microsoft 
Windows®.  Guide®  is  a  frame  based  authoring  system  with  a  dedicated  programming 
language,  Logiix®,  and  is  capable  of  opening  several  documents  simultaneously.  It  will 
import  and  export  Rich  Text  Files  (RTF),  maintaining  all  text  formatting  i:a  the  process 
It  can  import  several  graphics  file  formats,  both  bitmap  and  vector,  though  they  can 
only  be  imported  on  an  individual  basis,  not  embedded  within  a  word  processor  file. 
An  authoring  shell  along  with  authoring  macros  were  developed  using  Guide®  and 
Logiix®,  in  order  to  speed  the  authoring  process  and  to  encourage  widespread  use 
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among  the  faculty.  A  beta  version  of  the  system  was  implemented  in  the  fall  of  1991 
and  tested  with  140  medical  students  in  a  microbiology  course.  Early  testing  showed 
that  heavily  imported  text  could  be  imported  successfully,  the  authoring  process  could 
be  accelerated  through  the  use  of  maaos,  and  that  links  could  be  rebuilt  automatically 
after  documents  were  revised  using  word  processors  (Jackson  &  Duling,  1991).  After 
additional  revisions,  version  1,0  of  the  authoring  shell  was  used  by  two  faculty 
members  to  develop  questions  and  links  to  handouts  for  140  students  in  a  physiology 
course.  Cv-.rently  the  system  is  being  used  by  seven  faculty  members  in  two  courses 
using  four  different  .vord  processors.  The  system  is  heavily  utilized  and  has  received 
an  enthusiastic  response  from  the  students  in  both  formal  and  informal  evaluations. 

The  authoring  shell  developed  in  Guide  3.0®  includes  hypertext  document 
templates  for  questions  and  course  handouts,  a  standardized  interface  for  the  student 
users,  and  a  set  of  macros  for  faculty  authors  that  speed  the  creation  of  hypertext  links. 
The  macros  create  feedback  in  pop  up  windows  and  'reference'  links  to  the  course 
handout  documents  through  a  simple  dialog  process.  Faculty  authors  typically  require 
about  thirty  minutes  or  less  of  training  before  being  able  to  author  their  first  set  of 
hypertext  practice  questions.  Most  importantly,  the  macro  for  building  links  to  the 
handouts  uses  a  technique  that  automates  the  reconstruction  of  hypertext  links  after  a 
document  is  revised.  After  the 
faculty  authors  update  their 
handouts  each  year,  they  give  their 
new  word  processor  documents  to 
the  system  administrator  who 
imports  them  into  Guide®,  and 
executes  a  macro  in  the  Guide®  shell 
that  rebuilds  the  links  to  the 
questions. 

The  logic  of  the  macro  that 
creates  rebuildable  links  to  the 
handouts  is  relatively  simple.  The 
pseudo  code  is  included  in  Figure  1. 
The  author  must  have  a  document 
open  that  he  wants  to  link  to  before 
executing  the  maao.  Upon  clicking 
on  the  "Make  Reference"  button  the 
macro  creates  a  '.eference'  button  at 
the  top  of  the  question  frame,  then 
asks  "What  document  do  you  want 
to  link  to?"  The  macro  next  opens 
the  document  the  author  specifies 
and  asks  "Search  for  what  term?" 
After  the  author  enters  a  search 

string  and  clicks  an  "OK"  button,  the         p^-^^,^  j  p^^^^^  ^^^^  macro. 
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Make  Reference  Button  macro 

docA=  question  document 
docB=  handout  document 

Begin 

Create  Reference  button  at  top  of  frame  in  docA 
docB=Ask  "What  document  do  you  want  to  link  to?" 
Open(docB) 

searchterm=Ask  "Search  for  what  term?" 
hitword=Find(searchterm) 

On  finding  searchterm  Ask  "Link  to  this  word?"  Yes/No 
If  No,  repeat  Find(searchterm) 
If  Yes 
begin 

Make  hitword  a  reference  button 

Set  hitword  to  red  and  italic 

Complete  link  from  docA  reference  button 

hit+10=hitword  plus  the  following  ten  characters 

Begin  link  from  reference  button  in  docB 

Open  docA 

Insert  hit+10  into  top  of  frame  of  docA 
Set  text  color  of  hit+10  to  white 
Complete  link  to  reference  button  in  docA 
End  if 
End 
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macro  searches  through  the  target  document  until  it  finds  the  word,  highlights  the 
word,  then  asks,  "Link  to  this  word?"  If  the  author  clicks  "No,"  the  macro  searches  to 
the  next  instance  of  the  term  and  repeats  the  process.  When  the  author  clicks  "Yes"  the 
macro  makes  that  instance  of  th(i  word  a  'reference'  button,  sets  the  text  style  to  red  and 
italic,  and  completes  the  link  to  the  'reference*  button  in  the  question  document.  Then 
the  macro  puts  the  search  term  plus  the  next  ten  characters  that  follow  it  in  the  handouts 
document  into  a  string  variable.  The  macro  also  begins  a  link  in  the  handout  document. 
Next  it  reopens  the  question  document  and  inserts  that  string  variable  into  the  original 
question  frame,  setting  the  text  color  to  white  so  that  it  will  not  be  visible  against  the 
white  background.  The  macro  completes  the  link  from  the  notes  document,  creating 
two  hypertext  nodes  that  cross  reference  each  other.  The  actual  code  contains  many 
other  elements  and  error  handling  routines  that  are  not  included  here  since  they  are  not 
the  topic  of  discussion. 


Rebuilding  Hypertext  Links 


Faculty  authors  typically  revise 
processors  of  their  choice.  Once  they 
have  completed  their  revisions,  the 
formatted  and  revised  handouts  are 
converted  to  Rich  Text  Files  (RTF), 
and  imported  into  a  Guide® 
template.  The  system  administrator 
executes  anotlier  macro  that  rebuilds 
the  hypertext  links  between  the 
question  documents  and  the  new 
course  handouts,  which  have  no 
hypertext  links  initially.  This  macro 
goes  to  each  question  frame,  extracts 
the  search  term  plus  ten  characters 
string,  opens  the  new  course 
handouts,  searches  for  the  search 
term  plus  ten,  rebuilds  the  cross- 
referencing  links,  goes  to  the  next 
question  frame,  and  repeats  the 
process  until  all  the  hypertext  links 
have  been  rebuilt  See  Figure  2  for 
the  pseudo  code  of  this  macro. 
Typically  this  procedure  can  rebuild 
around  95%  of  all  links 
automatically.  The  macro  creates  an 
error  log  of  all  of  those  links  it  cannot 
rebuild,  due  to  the  target  area  having 
been  edited  by  a  word  processor,  so 


their  course  handouts  annually  with  the  word 


Rebuild  Links  macro 

docA=:question  document 
docB=handout  document 

question  document  must  be  open  to  first  question  frame 
Begin 

hit+1 0=:Get  hidden  string  at  top  of  docA  question  frame 
Begin  link  from  reference  button  in  question  document 
docB=Ask"Rebuild  links  to  what  document?" 
Open(docB) 
result=Find(hit+10) 
If  cannot  find  hit+1 0 
begin 

put  question  number  and  hit+10  into  error  log 
Answer  "Can't  rebuild  link!"  OK/Cancel 
else 

originalJerm=Drop  last  ten  characters  from  result 
Make  original Jerm  a  reference  button 
Set  text  of  reference  button  to  red  and  italic 
Complete  link  from  question  document 
Begin  link  from  docB 
Open{docA) 

Complete  link  from  docB  to  question  reference  button 
end  if 

Go  to  next  question  frame 
Repeat  until  all  questions  relinked 


Figure  2.  Pseudo  code  for  macro  to  rebuild  links 
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that  those  links  can  be  manually  reconstructed  at  a  later  time. 


Core  Logic 

The  core  logic  of  the  above  macro  is  a  very  simple  procedure  that  can  be  applied 
v/ithin  many  hypertext  authoring  systems.  When  a  reference  is  made  from  one 
document  to  another,  or  v^ithin  a  document,  the  target  term  plus  the  next  ten  characters 
that  follow  it  are  saved  in  an  indexed  area.  (Through  extensive  testing  with  20-50  page 
documents,  the  author  determined  that  the  target  plus  the  next  ten  characters  was  the 
shortest  string  that  is  extremely  likely  to  be  unique  within  any  document.)  The  target 
document  may  be  revised  using  a  word  processor  and  imported  anew  into  the 
hypertext  system,  lacking  any  hypertext  link  information.  A  macro  can  use  the  indexed 
target  term  plus  the  next  ten  characters  to  search  for  the  sites  of  the  original  links.  Once 
those  sites  are  located  the  links  can  be  rebuilt  automatically.  The  requirements  of  an 
authoring  system  that  can  execute  this  type  of  logic  include:  a  robust  programming 
language  with  text  string  handling  capabilities,  the  ability  to  open  several  documents 
simultaneously,  and  the  option  of  storing  strings  in  a  data  array  or  some  other  indexing 
system.  Most  hypertext  authoring  systems  can  meet  these  minimal  requirements. 

Limitations  of  the  Approach 

This  approach  to  rebuilding  hypertext  links  is  not  appropriate  in  some 
circumstances.  If  the  hypertext  application  contains  a  large  percentage  of  links  to 
graphics,  rather  than  text,  this  approach  will  be  of  little  use.  If  the  target  documents  are 
radically  rewritten,  changing  a  large  percentage  of  the  text  in  the  hypertext  target  areas, 
this  approach  will  not  be  able  to  rebuild  many  links  automatically.  Links  will  have  to 
be  built  manually  if  the  source  document  undergoes  editing  that  requires  referencing  a 
different  area  in  the  target  document.  This  system  is  optimal  when  two  documents  are 
cross  referenced  to  one  another  and  only  one  undergoes  significant  periodic  editing. 

Advantages  of  the  Approach 

The  advantages  of  this  approach  fall  into  two  categories:  those  applying  to  the 
shell/macro  based  system  as  a  whole,  and  those  that  apply  to  the  automated  hypertext 
link  rebuilding  system  alone. 

The  primary  advantage  to  the  shell /macro  based  system  is  that  content  experts  can 
use  the  word  processors  that  they  are  most  comfortable  with  when  creating,  editing, 
and  formatting  documents  rather  than  having  to  learn  yet  another  software  package. 
Although  this  may  seem  insignificant,  in  many  cases  this  factor  will  determine  whether 
teaching  faculty  will  take  on  the  additional  responsibility  of  authoring  and  maintaining 
a  hypertext  system.  Another  advantage  in  this  implementation  is  a  shell  that  makes 
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interface  design  decisions  for  authors  who  are  not  trained  in  software  interface 
development.  A  more  critical  advantage  is  that  all  routine  links  are  built  through  a 
dialog  process  rather  than  a  scripting  process  so  authors  can  concentrate  on  content 
rather  than  mastering  a  programming  language.  Authors  do,  however,  still  retain 
access  to  the  full  authoring  features  of  the  hypertext  system,  and  are  not  limited  to  the 
use  of  the  macros  alone.  The  use  of  macros  to  automate  routine  tasks,  such  as  creating 
links  within  and  between  documents,  speeds  development^  relieves  the  authors  of 
repetitive  tasks,  and  reduces  clerical  errors.  Since  the  syj  is  fast  and  easy  to  use, 
more  faculty  are  likely  to  begin  authoring  hypertext  o/Stems.  Perhaps  most 
significantly,  since  a  macro  rebuilds  nearly  all  of  the  hypertext  links  after  document 
revisions,  the  facnJiy  have  no  involvement  in  hypertext  document  maintenance  and  the 
system  administrator  has  only  a  minimal  time  investment.  Overall  the  system  removes 
about  95%  of  the  repetitive  tasks  that  make  up  hypertext  development  and  maintenance 
in  our  project.  The  savings  on  other  projects  will  be  determined  by  the  project  design 
and  demands,  but  the  approach  should  prove  useful  in  many  situation^^. 


Conclusions 

The  approach  described  can  reliably  rebuild  up  to  95%  of  the  iiypertext  links 
between  two  documents  when  one  has  been  significantly  edited  using  any  word 
processor.  The  core  logic  of  this  approach  can  be  used  with  many  hypertext  authoring 
systems  and  is  useful  in  many,  but  not  all,  situations.  The  approach,  combined  with  an 
authoring  shell/macro  system,  radically  accelerates  the  hypertext  authoring  and 
revision  process  making  it  more  efficient  and  more  attractive  to  prospective  faculty 
authors.  In  general,  the  approach  is  well  suited  to  the  development  of  hypertext  based 
CBI  developed  by  faculty  authors  with  a  minimum  of  computer  experience. 

This  case  study  points  out  several  areas  that  need  further  investigation.  Should  this 
approach  to  rebuilding  hypertext  links  be  integrated  into  the  hypertext  authoring 
systems  themselves  rather  than  as  macros?  Besides  making  the  authoring  process  more 
efficient  for  content  experts,  what  would  the  implications  be  of  designing  authoring 
systems  to  emulate  a  variety  of  word  processors?  Would  it  be  possible  to  integrate  this 
approach  with  the  automatic  text-to-hypertext  systems,  including  those  based  on 
SGML?  Ideally  one  system  would  allow  the  author  to  choose  between  several 
methodologies  of  making  and  rebuilding  hypertext  links  based  upon  the  structure  and 
needs  of  the  final  application. 
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Navigation  Maps  and  Cognitive  Styles  in  Hypertext  Assisted  Learning 
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Abstract 


This  paper  presents  the  results  of  a  study  which  iavesti gated  the  interactions  between  cognitive  styles  (learning  styles 
and  spati^d  ability)  and  navigation  maps  in  hypertext  assisted  learning.  Learning  outcome  and  user  satisfaction  were  used  as 
dependent  variables  in  an  analysis  of  the  interaction  of  learning  styles  and  menial  maps  in  the  design  of  hypertext  documents 
used  for  exploratory  learning. 

Three  research  questions  were  addressed:  (1)  Docs  cognitive  style  (learning  style  or  spatial  ability)  affect  learning 
outcome  or  user  satisfaction  in  hypertext  assisted  learning?  (2)  Docs  the  presentation  method  of  navigational  maps  affect 
learning  outcome  or  user  satisfaction  in  hypertext  assisted  learning?  (3)  Are  there  interaction  effects  h  -twccn  cognitive 
styles  and  method  of  presentation  which  affect  learning  outcome  or  user  satisfaction  in  hypertext  assisted  learning? 

A  total  of  282  students  enrolled  in  an  introductory  computer  science  course  volunteered  for  the  study.  Students 
completed  a  spatial  ability  test,  and  were  categorized  into  one  of  Kolb's  four  ieaniing  styles.  Tlie  students  were  sequentially 
assigned  to  one  of  four  treatment  groups,  each  using  different  navigation  methods  in  a  hypertext  document.  Students  spent 
one  hour  learning  as  much  of  the  text  in  the  docimient  as  possible.  Upon  reading  the  document,  students  completed  a 
questionnaire  which  measured  the  amount  of  material  learned,  and  the  level  of  satisfaction  in  using  the  lesson. 

An  ANOVA  procedure  was  used  to  comiparc  tlie  means  ofcognilive  styles  between  the  various  presentation  metliods. 
Analysis  of  the  data  produced  several  results.  (1)  Significant  differences  among  learning  outcomes  or  satisfaclion  belwecn 
learning  styles  were  not  found.  (2)  Metliod  of  presentation  indicated  significaiu  differences  in  both  Icitniing  outcome  mid 
user  satisfaction  level.  (3)  Spatial  ability  of  learners  produced  significant  differences  in  general  satisfaction  and 
disorientation  effects.  (4)  There  were  interaction  effects  involved  between  learning  styles  and  presentation  method  wliich 
had  a  significant  effect  on  user  satisfaction,  but  not  on  leartiing  outcome. 

Conclusions  from  tliis  study  indicate  that  while  there  are  differences  in  some  aspects  of  cognitive  style,  such  as  spatial 
ability,  it  could  not  be  detennined  that  learning  style  affects  either  learning  outcome  or  satisfac 'on  in  the  use  of  a  hypertext 
lesson.  However,  the  design  of  navigation  methods  for  presenting  hypertext  material,  docs  significantly  affect  both  learaing 
outcome  and  the  level  of  satisfaction  in  using  such  a  method.  Given  the  fairly  small  percent  of  varijuice  cxphiincd,  additional 
factors  accounting  for  the  variation  in  learning  outcome  should  be  explored  in  future  research. 


Hypertext,  described  as  nonlinear  text  or  nonsequential  writing  (Conklin,  1987)  provides  a  dramatic  tool  for  enhancing 
computer-assisted  learning.  Lessons  prepared  with  traditional  computer-assisted  instruction  technologies  arc  limited  in  tlieir 
ability  to  address  the  needs  of  individual  learners.  With  ti^ditional  methods  the  developer  must  have  prograumied  the  lesson 
to  present  the  material  with  as  many  imaginative  paths  as  possible.  Instead  of  developing  lessons  for  the  typical  learner,  the 
use  of  hypertext  technologies  provides  a  "customizable"  lesson  that  tlie  learner  may  adapt  to  meet  individual  preferences. 
Matching  llic  presentation  method  of  a  lesson  with  a  student's  preferred  learning  style  can  enhance  Iciiniing  outcomes  and 
encourage  a  more  active  approach  to  learning  (Smith  and  Reziilli,  1984).  Throughout  this  pajx^r,  the  term  hypertext  assisted 
learning  (HAL)  refers  to  the  addition  of  hypertext  to  Uaditiona'  computer  aided  instniction  or  computer  assisted  learning. 

The  interest  in  using  hypertext  in  HAL  lies  in  the  ability  to  present  large  amoimts  of  knowledge,  and  the  ability  to 
motivate  students  to  seek  that  knowledge.  Students  are  more  likely  to  absorb  concepts  when  given  the  opportunity  to 
explore  and  experience  these  associated  concepts.  Early  works  in  the  hypertext  field  (Bush,  1 945  and  Nelson,  1974) 
postulated  lhat  students  using  hypertext  might  develop  a  more  tliorough  knowledge  of  the  subject  and  its  coimections  with 
other  areas  of  inquiry.  Mayes  et  al.  suggest  hypertext  improves  learning  because  it  focuses  attention  more  on  the 
relationships  between  ideas  than  on  memorizing  isolated  facts.  These  associations,  that  could  be  symbolized  through  tlic 
linking  capabilities  in  hypertext,  facilitate  remembering,  concept  formation,  and  understanding.  Also,  tliere  is  the  possibility 
that  the  greater  amoiml  of  control  over  the  reading/learning  process  may  increase  involvement  and  desire  to  read  and  leani 
more  (Mayes  et  al.,  1990).  A  study  that  found  persons  prrferred  a  computer-based  hy|>erlext  presentation  of  infonnation  over 
a  paper  version  confirmed  this  enhancement  in  learning  (Shneiderman,  1989).  The  challenge  is  to  design  the  hypertext 
presentation  of  infonnation,  watli  the  associated  links,  to  offer  options  that  best  match  an  individual  students'  preferred 
learning  style. 

Hypertext  enhanced  learning  is  different  from  text  based  CAl  in  that  nonlinear,  linked  topics  can  be  presented  one  way 
by  the  author,  and  then  viewed  ^fferenlly  by  the  learner.  A  primary  suength  of  hypertext  is  the  ability  to  browse,  in  a 
nonlinear  style.  Although  an  author  can  group  topics  and  tlie  interlinkages  of  ideas  in  one  maimer,  tlie  reader  can  examine 
ideas  in  a  manner  better  suited  to  individual  preferred  modes  of  exploratory  learning  (Shneideniian,  1989).  However,  this 
suuctural  diversity  creates  a  problem  with  user  disorientation  and  cognitive  overhead,  knowing  what  infonnation  is  available, 
how  to  get  there,  and  remembering  where  previous  information  is  (Conklin,  1987).  Thus,  the  very  characteristics  of 
hypertext  that  often  make  it  desirable  also  cause  learning  problems  for  some  students.  One  solution  for  these  students  may 
may  be  a  guide,  in  the  form  of  a  graphical  map,  to  make  hypertext  more  effective  for  a  wide  diversity  of  leaniiiig  styles. 
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DIsotientotion  is  (he  tendency  to  lose  one's  sense  of  location  and  direction  in  a  nonlinear  document  (Conklin,  1987). 
The  problem  of  disorientation  is  a  function  of  having  to  know  where  you  are  in  a  network  of  associative  pallis»  and  how  to 
get  to  some  other  place  that  you  know  (or  think)  exists  in  the  network.  Hypertext  provides  multiple  paths  in  which  to 
move,  increasing  tlie  potential  for  the  user  to  become  lost  or  disoriented  while  searching  for  information.  Information  can 
easily  become  hard  to  find  or  even  forgotten  altogether.  A  sohition  for  coping  with  disorientation  is  to  use  graphical  maps, 
often  called  browsers.  Browsers  rely  on  the  higWy  developed  visuo-spatial  abilities  of  the  human  visual  and  information 
processing  system  to  help  create  <uid  store  a  mental  map  of  llie  pathways  in  a  hyi>ertext  system  (Conklin,  1987). 

Cognitive  oveiiiead  is  tlie  additional  effort  aiid  concentration  necessary  to  maintain  several  mental  tasks  at  one  time 
(Conklin,  1987).  The  problem  of  cognitive  overhead  occurs  while  reading  hypertext,  when  presenting  the  reader  with 
alternative  paths  to  follow.  These  choices  engender  a  certain  overhead  in  decision  making  that  is  absent  when  the  author 
already  has  made  many  of  these  choices  for  you.  The  problem  can  be  eased  by  having  a  graphical  map  or  browser  showing 
the  local  subnetwork  to  which  the  link  leads,  and  where  tiie  link  originated. 

While  the  use  of  hypertext  has  received  considerable  attention  for  the  advantages  it  affords,  all  students  are  not  assured 
of  benefiting  from  such  a  method  of  learning.  Ench  student  has  an  individual  learning  style  that  may  not  benefit  from  the 
use  of  hypertext  browsing.  While  some  students  prefer  to  leam  by  observation  through  listening  and  watching,  other 
students  prefer  to  leani  with  a  more  'hands-on'  approach  of  experimentation.  The  designer  of  a  hypertext  assisted  lesson 
could  include  a  learning  aid  that  would  match  the  different  learning  styles  of  students.  David  Kolb  (1984)  developed  an 
inventory  of  four  primary  learning  styles  that  categorize  persons  as  one  of  four  individual  learning  types. 

In  addition  to  the  individual  nature  of  learning  styles,  a  major  aspect  of  cognitive  styles  related  to  the  presentation  of 
hy[)ertext  learning  aids  is  the  spatiid  visualization  ability  of  individuals.  The  cognitive  overhead  required  in  the  navigation 
through  a  hypertext  document  places  a  demand  on  the  visuo-spatial  ability  of  the  reader.  This  study  addresses  the  interaction 
between  these  cognitive  styles  (Iciiniing  style  and  spaliiil  abilities)  and  tlie  use  of  hypertext  in  exploratory  Icaniing  of 
infonnation. 

The  piupose  of  this  study  was  to  investigate  a  user's  ability  to  assimilate  and  use  a  mental  map  of  information  in  order 
to  navigate  hypertext  enhanced  computer  aided  learning.  Tliis  study  assessed  the  relationship  between  cognitive  style 
(Kolb's  LSI  and  spatial  ability)  and  various  navigations^  mapping  techniques,  and  their  effect  on  learning  outcome  and 
learner  satisfaction  in  a  hypertext  environment.  Learning  outcome  and  the  subjective  nature  of  the  user's  satisfaction  were 
measured  with  a  posttest  questionnaire. 

This  research  seeks  to  answer  the  general  question;  do  individual  cognitive  styles  or  hypertext  navigation  methods 
affect  learning  outcomes  or  learner  satisfaction?  The  general  research  question  can  be  replaled  with  the  following  S|xjcific 
questions: 

( 1 )  Docs  an  individual's  cognitive  style  (learning  style  and  spatial  ability)  affect  learning  outcome  or  satisfaction  level? 

(2)  Do  navigational  maps  presented  in  a  hypertext  learning  system  affect  learning  outcome  or  satisfaction  level? 

(3)  Are  there  interaction  effects  between  cognitive  style  and  navigational  presentation  methods  providctl  on  learning 
outcome  and  learner  satisfaction? 

Overview  of  the  study 

A  document  with  material  to  be  learned  was  created  usin^  four  different  presentation  metliods  (Appendix  A).  Each 
version  contained  the  same  data  and  operated  under  the  same  hardware  and  software  platfonn.  HyperCard®  on  llie 
Macintosh™  computer  was  usetl  to  provide  a  tool  where  infonnation  could  be  presented  in  a  variety  of  computer  fonnats. 

The  Kolb  learning  style  inventory  mid  pai>cr  folding  test  provided  a  means  of  aitegori/ing  students'  cognitive  styles. 
Subjects  were  sequentially  assigned  to  one  of  four  presentation  methods  basal  on  their  Jeaniing  style.  Students  sjxsnt  one 
hom-  Iciiniing  material  using  one  of  the  four  presentatioji  methods.  A  posttest  on  the  material  measured  tlie  qu;uititative 
learning  outcome,  and  a  questionnaire  addressed  the  subjective  natiu-e  of  learner  satisfaction. 

The  deixjndent  variables  used  in  the  study  were:  (a)  an  objective  measurement  of  learning  outcome  and  (b)  subjective 
measurements  of  leamei  satisfaction.  The  independent  variables  include:  (a)  the  cognitive  style  of  the  leanier,  as  measured 
by  Kolb's  LSI  and  spatial  ability  measured  with  the  Paper  Folding  Test  and  (b)  mediods  of  presentation  of  material.  Tliere 
were  four  presentation  methods  used:  linear  text  only,  hyjxirtext  without  a  browsing  map  available,  hypertext  widi  a  textual 
navigation  map  available,  and  hypertext  with  a  spatial  navigation  map  available.  Univariate  analyses  of  variance  and 
covariance  were  used  to  test  for  significaiu  differences  in  the  means  of  the  learning  outcome  scores  and  satisfaction  levels 
between  the  different  presentation  methods  and  the  individual  cognitive  styles.  Specifically,  a  two-factor  ANOV A  was  used 
to  invcsligate  presentation  method  and  learning  style  with  spatial  ability  specified  as  a  covariate. 

Literature  Review 

The  use  of  hypertext  systems,  as  an  aid  in  enhancing  computer  aided  learning  systems,  is  the  focus  of  this  research,  llie 
foundation  for  diis  study  is  assessing  individual  cognitive  style  differences  as  measured  by  learning  styles  and  spatial 
ability,  and  hypertext  navigation  methods.  This  section  reviews  previously  reported  studies  and  developments  in  computer 
aided  instruction,  hypertext  systems,  learning  styles,  and  spatial  ability. 
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Computer  Aided  Instruction 

An  increasing  number  of  inicroconipulers  available  lo  sludenls  and  schools  shifted  tlic  cnipluisis  from  drill -«uid-pnicticc 
10  indqxjndcnt  learning  in  a  self-laughl  cnvironinenl.  A  review  of  the  lileraliire  revealed  a  number  of  research  studies  thai 
utilized  a  cognilive  or  learning  style  assessment  tool  and  a  CAI  treatment  (Benbasal  and  Taylor,  1982;  RoLcy  and  Taggarl, 
1981).  Many  previous  research  efforts  have  concenU*ated  on  the  use  of  cognitive  style  on  system  design  issues.  ITiesc 
studies  have  examined  learning  style,  field-dependency,  and  personality  types  such  as  the  Meyers-Briggs  type  indicator. 
This  line  of  research  has  found  oidy  limited  success  and  llic  findings  have  been  inconclusive. 

Rescaich  into  individual  differences  that  influence  learning  via  computer  systems  has  looked  mainly  at  educational 
backgrounds  and  issues.  While  cognitive  factors  have  often  been  unable  to  predict  learning  |>crfonnance  by  thcmsch  es,  they 
were  influencing  factors  in  conjunction  with  u*aining  nielliods  (Bostrom  et  al.,  1987).  In  Uieir  study,  Bostrom  el  al. 
predicted  a  strong  interaction  between  a  user's  characteristics  and  the  conceptual  model  used.  Tliere  is  considerable  debate  in 
education  literature  regarding  the  success  of  research  into  matching  or  tailoring  instructional  melliods  to  suit  individual 
needs.  Many  findings  indicate  thai  high  ability  subjects  do  well  regardless  of  the  method,  while  less-able  subjects  will 
benefit  from  tailoring  the  instnictional  approach  (Cronbach  and  Snow,  1977). 

Development  of  hypertext  learning  systems 

Tiie  origins  of  hypertext  may  be  traced  to  Vaiuievar  Bush  { 1 945)  who  predicted  the  problem  of  information  explosion. 
He  proposed  Memex,  a  machine  that  would  allow  users  reading  one  document  lo  switch  to  associated  material  in  another 
dceument.  Bush  based  this  concept  on  the  assumption  that  human  idea  processing  occurs  through  omnidirectional 
association.  Memex  would  provide  a  metliod  of  storing  and  reuicving  iiii brmalion  lo  support  hum<ui  inlomialiou  processing 
more  effectively. 

The  lenn  "hypertext"  first  appeared  in  the  writings  of  Ted  Nelson  (1974)  who  projx)sed  crciiting  a  uiiifial  on-line 
system  that  would  contain  one's  entire  literary  holdings.  This  system  would  only  be  praclicid  by  tlie  use  of  ins  concept  of 
using  the  nonsequential  chiiracteristics  of  hy|Krtcxt.  He  envisioned  hypertext  as  zdlowing  readers  to  create  j>ersonal  browsing 
paths  through  information  based  on  their  own  interests. 

Although  there  are  many  hypertext  systems  aviulable  today,  most  have  expanded  on  llie  ciulier  text-only  versions  juid 
now  provide  graphics,  soiuid,  video,  animation,  and  programming  capability.  One  way  to  describe  a  hyi)ertexl  system  is  to 
envision  a  database  with  a  node-link  network  added  lo  it.  Each  node  is  aniUogous  lo  a  window  on  a  computer  screen.  A 
link  is  a  word  or  icon  highlighted  in  such  a  way  that  it  provides  tJie  reader  with  fonvard  or  backward  branching  to  other 
nodes.  When  llie  reader  activates  a  link  by  selecting  it  with  a  mouse  or  a  funclior  key,  a  new  window/node  appears  with  the 
taiget  infonnalion  (Conklin,  1987).  Unlike  traditional  linear  text ,  such  as  mfonnation  on  paper,  or  hieraicliical  text  like 
chapters  and  sections  of  a  textbook,  hypertext  may  be  organized  so  that  the  reader  may  move  from  one  node  (a  module  of 
information)  to  another  along  any  number  of  paths  aroiuid  the  web  by  activating  a  visible  link.  Tliis  technique  allows  for 
what  Carlson  (1990)  calls  "three-dimensional  navigation  through  the  data."  These  non-linear  links,  via  hypertext,  provide 
infonnalion  in  themselves  through  the  associations  tlic  reader  creates  about  the  different  noiles. 

Carlson  (1990)  classifies  hypertext  applications  according  lo  the  cognitive  activities  that  Lhey  supjwrl,  reading  zuid 
learning.  Reading  is  goal-oriented  infonnation-seeking  navigation  through  a  large  imstruclured  library  of  iiifonnation  via 
browsing  tlirough  text.  Lxaniing  is  the  use  of  hyjperlext  lo  accommodate  varying  learning  acUviiies,  v«u-ying  the  speed  and 
stnictiu'e  of  navigating  the  material  in  a  personalized  slnicliu-e.  Hiis  study  investigates  the  use  of  hyj>erlcxl  in  a  reading  for 
learning  enviromncnl. 

Tlie  relationship  between  hypertext  fuid  compuler  assisted  Ictirning  seems  contradictory  at  first  ghuice,  i  lyjKrtext  by 
nature  allows  readers  to  follow  links  freely  wherever  they  choose.  Most  CAL  programs  on  the  other  hmid,  iue  baseti  on  tlie 
principal  of  programmed  instruction  whereby  the  next  appropriate  piece  of  information  is  presented  to  the  le<uner  when  he/she 
has  previewed  prerequisite  infonnation.  Hie  most  significant  factor  in  applying  hy|)ertext  to  computer  assisted  learning  is 
giving  the  learner  control  over  the  choice  of  material  and  the  route  tlu^ough  that  material.  As  such,  resjXMisihility  for  the 
learning  process  is  relinquished  to  the  learner  (Mayes,  et  al.,  1990).  llie  emphasis  sliifts  IVom  tui  instruction  system  to  a 
learning  system,  thus  the  tenn  1  lAL  would  accm-ately  rej)resent  an  emphasis  on  learning  in  a  (.liscovery  or  exploratory  mode. 

Allliough  browsing  in  hypertext  learning  is  often  lliought  of  as  meandering  through  infonnation,  the  added  value  of 
curiosity  and  the  freedom  of  choice  can  not  be  undervalued  (Duchastcl,  1990).  The  Icanicr  has  the  freedom  to  follow  either 
a  specific  j)ath  for  a  focused  search  of  infonnalion,  or  an  even  less  fiKuscd  search  based  on  0|>en  curiosity.  HyiKrlext 
enhanced  sciirch  of  infonnalion  then  is  a  lechnologicid  answer  lo  jui  luiabridgcd  huniiui  curiosity  for  more  information. 

'ITie  added  dimension  of  freedom  of  choice,  or  lack  of  slniclure,  may  cause  problems  for  some  Iciiniers.  J  or  some 
le<UTiers,  the  linear  structure  makes  sense  and  is  easier  lo  use.  The  table  of  conlenls  serves  as  a  visu«d  image  ol'  the  slnieturc 
of  the  material  and  as  a  map  of  where  lo  go  next.  Not  only  docs  the  table  of  contents  serve  as  a  mental  map  of  the 
organization  of  the  c  nenls,  but  reading  a  given  page  in  a  book  id  so  establishes  a  reference  point  lo  where  the  reader  is  at 
any  given  time.  In  a     asl,  it  may  be  difficult  for  some  learners  lo  fonn  such  a  model  of  hy|>erlexl  base  a  structiu'es.  Iliis 
difficulty  causes  the    ^er  lo  become  disoriented  or  "lost  in  hypcrspace."  Disorientation  also  leads  lo  an  additional  problem, 
the  extra  mental  effort  required  of  the  learner  to  navigate  the  complex  stnicture  requires  cogmli  ve  overhead  (Conklin,  1 987). 

Textbooks  and  traditional  computer-aided  instruction  do  not  require  the  exu-a  cognitive  effort  required  of  hyi>erlexl 
docimienls.  The  requirement  of  additional  cognitive  capacity  may  mean  that  infonnation  processing  nonnally  <dlocated  to 
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thinking  about  the  material  would  be  used  in  the  processing  of  maintaining  the  hypertext  slruclurc  (Mayes  et  al.,  1990). 
Although  Mayes  suggests  that  the  use  of  hypertext  enhances  the  instructional  process,  the  cognitive  overhead  and 
disorientation  problems  may  offset  these  advantages  for  many  learners.  One  challenge  in  designing  hypertext  learning 
systems  is  determining  whether  the  cognitive  overhead  required  to  use  hypertext  is  a  significant  factor  in  the  effectiveness  of 
such  a  system.  Adopting  a  form  of  spatial  visualization  in  ilie  presentation  to  provide  a  navigational  aid  for  some  students 
could  onset  this  problem  by  indicating  conceptual  relalionsliips.  The  problem  with  providing  such  a  system  lies  in 
designing  an  interface  that  would  allow  for  a  wide  variation  in  visual  ability.  A  user  of  a  hypertext  enhanced  system  needs 
to  have  a  level  of  navigation  aid  consistent  with  his/her  visual  cognition.  Such  a  system  should  also  provide  the  ability  to 
change  the  level  or  complexity  of  detail  while  navigating  paths  (Mayes  ct  ai.,  1990). 

A  key  aspect  of  this  study  is  to  investigate  how  cognitive  style  (learning  styles  and  spatial  cognition)  interacts  with  the 
use  of  hypertext  learning.  For  example,  learners  with  high  spatial  ability  may  not  need  (or  prefer)  as  much  help  developing  a 
mental  map  as  those  witli  lesser  abilities. 

Learning  Styles 

The  knowledge  that  all  students  do  not  icam  in  the  same  manner  has  led  researchers  to  identify  students'  preferred 
learning  styles.  A  learning  style  is  part  of  an  individual's  cognitive  structure  and  refers  to  the  pattern  of  behavior  and 
performance  by  which  an  individual  approaches  educational  or  learning  experiences.  Carl  Jung  was  one  of  the  first  persons 
to  use  the  concept  of  personality  types  to  explain  the  different  ways  people  perceive  and  process  information  (Kolb,  1984). 
Jung  identified  a  person's  orientation  toward  the  internal  (introverted)  world  of  self,  or  the  external  (extroverted)  world. 
Jung's  second  asj^l  of  a  person's  orientation  with  the  enviroimient  was  to  analyze  how  he/she  perceives  input  from  the 
environment  Uirough  sensation  or  intuition.  A  third  aspect  was  the  way  of  processing  that  information,  or  judgement  about 
tliat  data  via  thinking  or  feeling. 

Jung's  work  has  served  as  the  foundation  for  many  theories  in  the  areas  of  personality  types,  cognition,  and  learning 
styles.  His  research  has  been  used  to  formulate  numerous  instniments  to  identify  learning  differences  among  individuals. 
Jung's  work  formed  much  of  Kurt  Lewin's  "Field  Theory"  which  suggests  that  human  behavior  is  a  fimction  of  a  person  and 
the  environment  (Kolb,  1984).  Lcwin  believed  that  learning  should  be  viewed  in  a  four-stage  cycle.  The  first  stage, 
immediate  concrete  experience,  forms  the  basis  for  the  second  stage,  the  observations  and  reOections  about  the  experience. 
The  third  stage,  the  formation  of  abstract  concepts  and  generalizations,  focuses  on  the  information  gained  in  stage  two  and 
yields  "theories"  from  which  new  implications  of  actions  can  be  deduced.  The  fourth  stage,  testing  these  theories  in  new 
sUuations,  guides  the  individual  to  new  experiences,  and  then  the  learning  cycle  begins  again,  only  this  time  with  the  added 
benefit  of  previous  experience  and  knowledge  (Kolb,  198*0- 

Lewin's  tlieory  had  a  major  influence  on  David  Kolb  as  he  developed  his  "th-jory  of  cx|Kriential  learning."  Kolb's 
theory  suggests  that  knowledge  is  developed  through  a  transformation  of  experiences  (Kolb,  1976, 1984,  1985).  Kolb 
believes  mat  these  experiences  require  an  individual  to  use  combinations  of  four  basic  "le^iniing  modes": 

1.  Concrete  Experience  (CE)  based  on  "feeling" 

2.  Refieclion  and  Observation  (RO)  based  on  "watching" 

3.  formation  of  Abstract  Conceptualizations  (AC)  based  on  "thinking*' 

4.  Active  Expcrimenlalion  (AE)  based  on  "doing" 

The  four-stage  learning  process  reveals  that  learning  rajuires  abilities  that  arc  polar  opjwsites,  therefore  the  learner  is 
forced  to  decide  which  set  of  learning  abilities  is  needed  to  apply  to  each  learning  situation.  When  the  four  learning  abilities 
(CE,  RO,  AC,  AE)  are  represented  in  the  experiential  learning  model  tiiey  form  two  axes  that  represent  the  two  primary 
dimensions  of  the  learning  process  (Smith  &  Kolb,  1986). 

The  individual's  "learning  style"  is  a  function  of  how  the  four  modes  are  used.  Kolb  places  these  four  modes  in  a  two 
dimensional  model;  the  active-reflective  dimension  and  the  concrete-abstract  dimension.  (Sec  Figure  1)  The  concrete-abstract 
dimension  places  abstract  conceptualization  on  one  end  of  the  learning  mode  and  concrete  experience  on  the  other.  This  scale 
indicates  how  an  individual  perceives  information.  Persons  who  prefer  to  sense  and  feci  their  way  through  new  experiences 
and  information  are  located  at  the  concrete  end  of  this  scale,  and  prefer  the  more  tangible  aspects  of  new  experiences  and 
information.  On  the  other  end  of  the  scale,  the  abstract  Ic^irner  prefers  to  think  about  their  cxi)crienccs  in  an  analytiud 
metliod  using  symbolic  methods.  (Sniilli  &  Kolb,  1986) 


The  active-reOective  dimension  places  active  experimentation  learning  mode  on  one  end  and  die  refieclive  observation  on 
the  other.  This  scale  indicates  how  an  individual  processes  die  informauon  perceived.  Kolb  believes  that  individuals  have  a 
favorite  mode  in  each  learning  dimension.  An  individual  with  a  score  in  the  "active"  portion  of  the  active-refieclivc  scale 
typically  prefers  lo  leani  in  a  hands-on,  learner  controlled  "doing"  environment.  A  person  with  a  score  in  the  "reficctivc" 
portion  normally  prefers  to  Icani  by  "watching"  such  as  in  a  lecture  setnng.  As  with  the  concrete-abstract  dimension,  persons 
will  be  located  throughout  the  active-refiective  scale,  and  no  score  is  idenufied  as  being  better  than  another  (Smith  &  Kolb, 
1986).  'Ilicrc  is  a  dominant  and  a  suppressed  mode  in  each  of  the  dimensions.  Although  an  individual  must  draw  on  e<ich  of 
the  four  modes,  when  in  a  learning  situation,  a  favorite  will  develop  among  the  four  mid  be  develoixid  further,  'lliis  is  what 


Insert  I^'igure  1  about  here. 
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he  calls  the  learning  style"  of  an  individual.  Kolb  (1984)  describes  the  characterisiics  of  llie  four  learning  styles  and 
environment  from  which  an  individual  develops  le^miing  skills  as  follows. 

Di vergers,  (low  AC-CE  and  low  AE-RO  scores)  The  dominant  learning  abilities  are  concrete  exjKricncc  and  reflective 
observations.  These  persons  tend  to  be  more  oriented  toward  observation  than  action.  The  cli  verger's  greatest  strength  lies  in 
imaginative  abihty  and  awareness  of  concrete  situatioi\s  from  many  |>erspectives.  This  learning  style  docs  better  in  instances 
thai  call  for  generation  of  ideas  such  as  a  "brainslonning"  ioea  session. 

Convergers,  (high  AC-CE  and  high  AE-RO  scores)  The  dominant  learning  abilities  arc  abstract  conceplujiiization  and 
active  expenmentations.  Tlie  gieatcst  strength  of  this  approach  lies  in  problem  solving,  decision  making  and  tlie  practical 
application  of  ideas.  This  style  seems  to  do  best  using  hypothetical  and  deductive  reasoning,  where  tlierc  is  a  single  correct 
answer  or  solution  to  a  question  or  problem. 

Acconimodalors.  (low  AC-CE  and  high  AE-RO  scores)  The  dominant  learning  abilities  arc  concrete  experience  and 
active  experimentations.  Their  greatest  strength  lies  in  doing  things;  carrying  out  plans  and  experiments  and  involving 
themselves  in  new  experiences.  They  prefer  new  experiences  and  onportunities.  FVobiems  lend  to  be  solved  in  an  intuitive 
trial -and-enor  method,  relying  on  other  people  for  information  rauicr  than  using  their  own  analytical  abilities. 
Accommodators  excel  in  those  siniaiions  where  they  must  adapt  to  specific  immediate  circumstances. 

Assimilators.  (high  AC-CE  and  low  AE-i^O  scores)  The  dominant  learning  abilities  are  abstract  conceptualization  and 
reflective  observations.  Their  greatest  strength  lies  in  inductive  reasoning  and  the  ability  to  create  theoretical  models.  This 
style  is  less  focused  on  people  and  more  concerned  with  ideas  and  abstract  c^jncepls. 

The  Learning  Style  Inventory 

Kolb  developed  an  instrument  to  measure  where  an  individual  falls  on  the  two  (AC-CE  and  AE-RO)  dimensions;  tlic 
Learning  Style  Inventory  (LSI).  Using  both  linear  scales,  ilic  intersection  of  the  scores  places  an  individual  in  one  of  the 
four  qu^rants  in  Kolb's  LSI.  (See  Figure  1)  Those  in  the  concrete-active  portion  arc  referred  to  as  "accommodators."  In  the 
concrete-reflective  quadrant,  individuals  are  referred  to  as  "divergcrs."  If  the  intersection  of  scores  as  in  the  active-abstract 
quadrant,  individuals  are  a  "converger,"  and  in  the  abstract -reflective  quadrant  the  lenn  "assiniilator"  is  used  to  characterize 
learning  style  preference. 

Kolb  Ctvs^  developed  his  Learning  Style  Inventory  in  1976  and  revised  it  in  1985.  Using  Kolb's  instrument,  responses 
to  twelve  sdf -administered,  self-scoring  questions  can  be  tabulatal  to  detenninc  a  score  for  each  ol"  the  four  learning  modes. 
The  1985  instrument  has  twelve  sets  of  four  self-descriptive  sentence  endings,  corresponding  to  the  four  learning  modes. 
Subjects  rank  endings  for  each  sentence  from  1  to  4,  where  a  1  would  be  least  descriptive  of  one's  leaning  style  and  a  4 
would  best  characterize  one's  learning  style.  Scores  are  tlien  smnnicd  for  each  colunui  (sentence  ending)  of  each  statement, 
giving  a  total  for  each  learning  mode.  To  determine  preferred  learning  style,  these  four  scores  are  combiner'  oy  taking  the 
abstiact  conceptualization  (AC)  score  and  subtracting  the  concrete  experience  score  (CE),  givitig  tlie  AC  JE  score, 
indicating  'thinking"  vs.  Tecling'  preference  of  learning.  The  active  experimentation  (AE)  minus  reflective  observation  (RO) 
score  provides  the  AE-RO  score,  that  indicates  'doing'  vs.  hatching'  preferences.  By  plotting  each  of  these  two  combined 
scores  on  the  LSI  grid  and  drawing  a  corresponding  line,  the  intersection  |K)int  generated  will  fall  in  one  of  the  four 
quadrants,  representing  an  individual's  dominant  learning  style. 

Validity  and  Reliability  of  the  I^I.  The  LSI  has  become  one  of  the  most  conuiionly  used  measures  of  learning  style. 
This  section  will  address  the  validity  and  reliability  of  the  LSI  iiistrimient.  The  1985  version  was  revised  to  answer 
criticisms  of  the  1976  instrument.  The  revised  v.^'  ion  doubled  liie  number  of  sets  to  be  ranked,  changed  from  ranking  four 
word  sets  to  completing  sentences  with  four  enoings.  These  changes  were  designed  to  make  the  instnunent  easier  to 
understand,  and  clarify  the  concepts  being  measured.  Kolb  used  a  larger  sample  to  establish  the  1985  statistics  (n=:268). 
Sinith  &  Kolb  (1986)  report  improved  reliability  in  the  1985  instiimieiil  with  a  good  intenial  reliability  for  the  four  learning 
modes  and  two  combined  scores  as  measuicd  with  Croiibacli's  alpha.  (Kolb,  1985). 

Suattai  Ability 

The  art  of  navigating  infoniiation  systems  is  compared  to  navigating  in  the  real  world.  It  is  necessciry  to  maintain  a 
sense  of  direction  within  an  infonnation  system,  much  as  one  must  develop  an  orientation  to  real  world  enviroiunents 
(Shum,  1990;  Bolt,  1979;  Canter,  el  al.,  1985).  The  ability  to  assimilate  and  inainlain  a  mental  image  of  navigational  paths 
is  directly  related  to  one's  spatial  ability.  Spatial  ability  refers  to  a  persons  ability  to  manipulate  or  transfonii  an  image  of 
spatial  patterns  into  other  arrangements  (Ekstroiii  et  al.,  1976). 

Hypertext  is  particularly  associated  with  spatially  demanding  tasks:  jumping  from  node  to  node,  following  a  path  in  the 
hypertext  web,  and  getting  lost  or  disoriented  while  browsing  a  hypertext  document.  A  person  keeps  an  orientation  with  a 
cognitive  map,  or  an  orienting  schema,  tliat  is  a  mental  representation  of  information  in  a  spatial  arrangement.  Downs  and 
Stea  (1973)  define  the  development  of  a  r  gnitive  map  where  "an  individual  acquires,  codes,  stores^  recalls  and  decodes 
information  about  the  relative  location  a  d  attributes  of  phenomena  in  his  ever)'day  spaUal  environment.'*  This  definition 
emphasizes  that  cognitive  m^s  contain  not  only  the  information,  but  also  the  attributes  about  'where'  that  infonnation  is, 
including  how  to  get  there  and  the  environment  around  the  'place'. 

Just  as  one  develops  a  mental  map  of  how  to  get  to  a  familiar  place,  such  as  a  local  school  or  store,  a  hypertext  re<ider 
also  forms  a  mental  map  of  how  to  get  to  a  particiUar  piece  of  infonnation.  Whether  or  not  the  development  of  such  a 
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cognitive  map  is  contingent  on  the  availability  of  a  visual  map  is  an  aspect  of  this  study.  Downs  and  Stea  (1973) 
distinguish  two  different  ways  by  which  hypertext  rtjaders  develop  this  spatial  information  --  direct  and  vicarious.  Direct 
source  of  information  refers  to  experiencing  and  personalizing  information  in  what  is  called  a  learning  by  doing*  method. 
Vicarious  sources  are  gained  by  second-hand  information,  such  as  a  graphical  map  or  narrative  description.   A  vicarious 
source  is  one  that  is  provided  by  someone  else  through  a  graphical  browser  or  graphical  map.  This  second  method  makes  it 
possible  for  a  learner  to  form  a  mental  map  of  what  is  available  without  having  to  visit  each  node,  made  possible  through  a 
visual  display  that  is  part  of  the  software  program.  An  example  of  this  idea  would  be  supplying  a  map  of  the  structure  of  the 
links  and  nodes.  Often  these  maps  function  as  an  interactive  menu  with  tlic  map  of  the  nodes  serving  as  buttons  to  link  the 
nodes  together  . 

It  is  a  natural  human  propensity  to  impose  structim;  on  ever>'day  concepts.  Just  as  it  is  natural  to  describe  everything  as 
"on  top  of...,**  or  "just  east  of etc.,  it  also  is  natural  for  human  processing  to  impose  structure  on  information  sources. 
Thus  in  a  hypertext  environment,  the  use  of  an  absolute  reference  (such  as  a  page  number)  would  not  be  as  efficient  as  using 
names  of  nodes  in  a  cluster  arrangement.  For  example,  in  the  hypertext  version  of  Jakob  Niclsen^s  Hypertext  '87  Trip  Report, 
(Nielsen,  1988)  a  reader  selects  one  of  several  topic  clusters,  ancf  then  a  particular  topic  under  that  cluster.  It  is  easy  then,  for 
readers  to  remember  that  they  selected  tlie  cluster  "Hypertext  Systems"  and  the  topic  "HyperCard." 

The  Pap«*  Folding  Test 

Spatial  ability  casi  be  evaluated  and  measured  with  a  variety  of  msiruments.  One  of  the  most  consistent  and  reliable 
instruments  used  in  recent  spatial  research  has  been  the  Paper  Folding  Test,  part  of  the  Kit  of  Factor-Referenced  Cognitive 
Tests.  The  spatial  orientation  and  visualization  tests  originated  fron?  Che  work  of  L.  Thurstone  and  H.  A.  Witkin  (Smith, 
1964;  Ekstroni,  ct  al.,  1976). 

The  visualization  and  spatial  orientation  measurements  offered  in  the  Kit  of  Factor-Referenced  Cognitive  Tests  measure 
mudi  of  the  same  spatial  abilities  as  the  cognitive  factors.  However,  the  visualization  test  requires  more  mental  restructuring, 
and  therefore,  reflects  a  more  significant  element  of  spatial  orientation.  Cattell  (1971)  suggests  that  the  visualization  factor 
test  is  actually  a  second-order  factor  and  includes  spatial  ability.  Carroll  (1974)  notes  that  tlie  visualization  test  requires  tlie 
subject  to  do  additional  serial  operations  in  addition  to  simple  mental  rotation  of  spatial  images  (Ekstrom,  ct  al.,  1976). 

The  Paper  Folding  Test  presents  the  subject  with  an  illustration. of  a  piece  of  paper  with  two  or  three  folds  (indicated 
by  dashed  lines)  and  with  a  hole  punched  in  it  after  the  last  fold.  The  subject  then  is  asked  to  select  one  of  five  drawings 
that  shows  the  correct  version  of  how  the  paper  would  look  like  when  unfolded  (Ekstrom,  ct  al..  1976).  This  test  is 
designed  to  measure  an  individual's  ability  to  visualize  and  manipulate  an  image  of  spaticU  patterns  (Ekstrom,  ct  al.,  1976). 

Reliability  of  tlie  Paper  t^olding  Test 

Statistical  data  provided  with  the  manual  for  the  Paper  Folding  Test  gives  reliability  figures  from  two  different  studies. 
A  study  of  46  college  students  reported  a  mean  Vz-2  score  of  13.8  and  a  Cronbach's  alpha  value  of  .84.  In  another  study  of 
82  Army  enlistees,  a  mean  score  of  10.4,  {dso  with  a  Cronbach's  alplia  value  of  .84  was  found  (Ekstrom,  el  ai.,  1976). 

Research  Method 

This  study  comparexl  the  interactions  of  cognitive  style  (as  measured  by  learning  style  and  spatial  ability)  and  the 
availability  of  navigation  maps  in  the  use  of  hy|)erlext  assisted  learning.  The  cognitive  styles  of  students  were  correlated 
with  various  presentation  methods  within  a  i)redcfined  map  of  information.  This  section  describes  (a)  the  general  research 
questions,  (b)  subjects,  and  their  assignment  to  an  ex]Krimcntal  group,  (c)  the  measurement  of  cognitive  style  (Kolb's  USl 
and  spatial  ability  Vz*2),  (d)  the  treatment  meiJiods  of  presentation  of  material,  (c)  the  measurement  of  both  the  objective  and 
subjective  data,  and  (f)  design  and  analysis  procedures,  'llic  resulting  design  is  represented  grapliically  by  Figure  2. 

Insert  I-igure  2  about  here. 

Genera]  Research  Questions 

Three  research  questions  were  addressed:  (1)  Does  cognitive  stvlc  (Icanung  style  and  spatial  ability)  affect  learning 
outcome  or  user  satisfaction  in  hypertext  assisted  learning?  (2)  Does  the  presentation  method  of  navigational  maps  affect 
learning  outcome  or  user  satisfaction  in  hypertext  assisted  learning?  (3)  Are  there  interaction  effects  between  cognitive 
styles  and  method  of  presentation  which  affect  leaniing  outcome  or  user  satisfaction  in  hypertext  assisted  learning? 

To  answer  the  research  questions,  this  study  investigated  the  interactions  between  cognitive  styles  and  navigation  maps 
in  hypertext  assisted  learning.  Cognitive  styles  were  measured  considering  two  aspects  of  interest  in  this  study;  learning 
styles  and  spatial  ability.  Learning  outcome  and  user  satisfaction  were  us^  as  dependent  variables  in  an  analysis  of  the 
interaction  of  learning  styles  and  mental  map  aids  in  (he  design  of  hypertext  documents  used  for  exploratory  learning. 
Table  1  provides  a  summary  of  variables  used  in  this  study.  The  research  questions  above  were  addressed  by  first  analyzing 
learning  outcome  scores,  and  then  analyzing  satisfaction  levels  for  each  of  the  questions. 

Insert  Table  1  about  here. 
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Hypotheses  and  Expected  Results 


The  expcrimmtal  design  described  in  the  following  scctious  will  be  used  lo  lest  the  null  hypotheses  listed  below.  The 
three  generSi  research  questions  are  restated  into  specific  statistical  hypotheses,  each  dealing  alternatively  with  learning 
outcome,  and  then  the  factor  scx^res  of  satisfaction  levels  for  each  general  research  question. 

Hl0.  Learning  Style  Differences.  There  are  no  significant  differences  in  outcome  measures  or  satisfaction  level 
among  different  learning  style  groups  of  students.  This  could  be  stated  asHl^:  /^j  -  h^2  ~/^3        where  ja^  represents 
the  mean  of  the  outcome  measure  for  one  of  the  foiu*  learning  style  groups.  Learning  style  differences  are  analyzed  as  a  main 
effect  in  an  ANOVA  analysis. 

If  rejected,  this  hypothesis  suggests  diat  students  with  different  leaniiug  styles  will  perform  differently  within  this 
study.  Specifically,  it  is  predicted  that  those  in  the  reflective  -  abstract  mode  (assimilators)  will  tend  lo  prefer  a  lecture  over 
a  computer,  and  thus  may  not  learn  as  well  with  tlie  material  presented  in  this  research. 

H20.  Spatial  Abilities.  There  are  no  significant  differences  in  the  means  of  learning  outcome  or  satisfaction  level 
among  different  spatial  abilities  of  students,  lliis  could  be  stated  as  Ul^:  Bj  =  0  where  Bj  represents  llie  linciu-  relationship 
between  spatial  ability  and  performance  on  tlie  dependent  variable.  Spatial  ability  is  measured  with  the  paper  folding  test 
and  analyzed  as  a  covariate  in  an  ANOVA  analysis. 

The  alternative  hypothesis  suggests  that  students  with  higher  spatial  abilities  will  tend  to  score  higher  on  the  objective 
outcome  measure,  and  will  be  more  satisfied  willi  a  hypertext  system,  than  those  with  low  spatial  ability. 

H30.  Pi-esentation  Methods.  The'e  are  no  significant  differences  in  the  means  of  learning  outcome  or  satisfaction 
level  among  different  presentation  ire  itment  method  groups  of  students.  This  could  be  stated  as  1O0:  }i^-}i2-  /'3  =  /<4 
where  /^j  represents  mean  of  the  outcome  measure  for  one  of  (he  four  treatment  method  groups  for  the  presentalion  of  the 
material.  Presentation  method  is  analyzccl  as  a  main  effect  in  an  ANOVA  analysis. 

It  could  be  expected  that  presentation  methods  three  and  four  will  produce  better  learning  outcomes,  if  for  no  otiier 
reason  than  it  helps  students  oi  all  learning  styles  to  create  and  use  a  structure  of  the  material. 

H40.  Interaction  Effects.  There  are  no  significant  interaction  effects  between  learning  style  and  treatment  ^nethod  in 
their  effect  on  learning  outcome  and  satisfaction  level.  The  interaction  of  learning  style  and  treatment  method  is 
H40:  AllO^jj  -  ;/j  -   j  +  ;<)  =  0  that  slates  for  all  treatment  groups,  when  you  subtract  the  effect  of  learning  style  (j4^  )  and 
the  effect  treaunent  method  O^j)  from  the  total  effect  (/^y),  and  then  add  in  the  total     the  remaining  value  should  be  zero. 
The  interactions  between  learning  styles  and  treatment  groups  are  analyzed  as  the  interaction  effect  in  an  ANOVA  analysis. 

It  is  expected  that  students'  learning  style  will  interact  with  a  particular  presentation  method  in  such  a  way  to  improve 
or  degrade  the  learning  performance  and  satisfaction  levels.  For  example,  those  students  on  the  reflective  end  of  the  AE-RO 
scale,  would  not  lean)  as  well  with  a  hypertext  system  as  much  as  with  a  sequential  linear  or  liierarcliical  presentation. 

Stibiect  population 

The  subjects  of  this  study  were  282  undergraduate  students  enrolled  in  introductory  Computer  Science  courses.  The 
selection  of  students  at  this  level  as  subjects  for  lliis  study  was  based  on  the  assiunption  that  they  would  have  at  least  a 
passing  interest  in  a  hypertext  lesson,  while  at  the  same  time  have  little  or  no  previous  exposure  to  the  topic.  Each  student 
completed  two  instruments  of  cognitive  style:  a  learning-style  inventory  (Kolb's  LSI)  and  a  paper  folding  test  for  spatial 
ability  (Vz-2). 

Measdrement  of  cognitive  style 

Kolb^s  LSI .  The  Kolb  Learning  Style  Inventory  (1985  version)  was  used  to  measure  each  subject's  dominant  le;ui)ing 
style.  The  1985  version  of  the  LSI  is  a  l2-question,  one-page  instrument  with  12  sentences,  each  with  four  different  ending 
statements  corresponding  to  the  four  learning  modes.  Each  set  begins  with  a  sentence  like  "I  learn  best  by:"  followed  by 
four  sentence  endings  such  as;  "a.  feeling,  b.  watching,  c.  thinking,  d.  doing".  The  subject  assigns  a  value  of  4  to  the  ending 
that  best  fits  their  preferred  method  of  learning.  The  remaining  three  are  assigned  a  value  of  3, 2,  or  I,  corresponding  to 
decreasing  values  for  their  preferred  method  of  learning. 

After  ranking  the  four  endings  for  all  12  sentences,  subjects  scored  the  LSI  by  totaling  the  values  assigned  for  each 
column.  Each  column  corresponds  with  one  of  the  four  preferred  learning  modes,  lliese  four  column  totals  indicate  the 
emphasis  each  student  places  on  the  four  learning  stages  as  an  approach  to  learning.  These  four  scores  were  tlien  combined 
to  show  the  respondent's  emphasis  of  abstract  conceptualization  over  concrete  expaicncc,  and  active  experimentation  over 
reflective  observation.  These  two  combined  scores  are  plotted  on  the  Learning-Style  Type  Grid.  The  intersection  point  of 
the  two  modes  will  fall  in  one  of  the  four  quadrants  and  identify  the  subject's  preferred  learning  style. 

The  Paper  Folding  Test.  The  Paper  Folding  Test,  a  variation  of  Thurstone's  "Punched  f  loles"  test,  measures  an 
individual's  spatial  ability.  A  piece  of  paper  is  presented  to  the  subject  witli  illustrations  of  two  or  three  folds  (indicated  by 
dashed  lines)  and  with  a  hole  punched  in  it  after  the  last  fold.  The  subject  then  is  asked  lo  select  one  of  five  drawings  that 
shows  the  correct  version  of  how  the  paper  would  look  like  when  imfolded.  The  student  had  Ihree  minutes  to  complete  the 
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first  10  drawings,  and  another  three  minutes  for  a  second  set  of  10  drawings.  The  score  of  the  Vz-2  was  calculated  by 
.adding  one  point  for  each  correct  selection,  and  subtracting  1/5  point  for  each  wrong  selection. 

Treatmeint  Methods 

There  were  four  groups  of  students  different  methods  of  presentation  of  information  (Refer  to  Figure  2).  All  four 
groups  were  provided  with  the  same  information,  differing  only  in  the  amount  of  hypertext  links  and  navigational  maps.  The 
material  presented  was  a  copy  of  Jakob  Nielsen's  'Trip  Report  of  Hypertext  *87."  (See  Appendix  A)  provides  sample 
screens  of  these  groups)  The  four  text  treatment  groups  are:  (1)  sequential  text  only,  with  only  a  page  forward,  page  back,  or 
return  to  the  table  of  contents  available.  (Appendix  A-1)  (2)  a  hypertext  version  of  the  report  with  buttons  avaifable  to  move 
to  other  parts  of  the  report,  but  without  mapping  (o  information  available.  (Appendix  A-2)  (3)  a  hypertext  version  of  the 
repcm,  with  a  textual  map  visible  on  the  screen  that  not  only  served  as  a  map,  but  also  as  a  menu  to  select  other  parts  of  the 
report,  (Appendix  A-3)  (4)  a  hypertext  version  of  the  report,  with  a  spatially  organized  map  visible  on  the  screen  that  served 
as  both  a  map  and  as  a  menu  to  select  other  parts  of  the  report.  (Appendix  A-4)  All  of  the  methods  were  presented  witii  an 
Apple  Macintosh  HyperCard  stack. 

The  HyperCard  stacks  were  modified  so  tliat  all  of  the  text  material  was  identical,  with  tlic  only  difference  being 
hypertext  navigation  options.  The  selection  of  this  stack  wjis  made  for  a  number  of  reasons.  First,  the  second  and  fourth 
treatment  methods  were  already  available,  witli  only  minor  modification  necessary.  Secondly,  the  material  in  die  report 
should  have  been  of  interest  to  computer  science  students,  who  had  litdc  or  no  previous  knowledge  of  the  subject  matter  - 
providing  approximately  equal  base  knowledge  level.  Random  assigimient  to  treatment  conditions  also  helped  ensure 
balance  of  initial  topic  knowledge.  A  posttest  only  design  eUrainated  pretest  sensitivity  potential  threat  to  the  intemal 
validity  of  the  study. 

Snblect  asstgyimcnt  to  trcatm>ent  method 

Upon  completion  of  the  cognitive  style  tests,  the  students  were  sequentially  assigned  to  one  of  four  different 
presentation  methods  after  they  were  classified  into  one  of  die  learning  style  groups.  This  method  of  assignment  assm^  an 
approximately  equal  number  of  subjects  from  each  learning  style  in  the  four  Afferent  treatment  groups.  A  total  of  2S2 
students  took  part  in  the  study.  Tlie  subjects  were  given  one  hour  to  complete  die  study.  This  time  allocation  was 
calculated  from  a  pilot-test  conducted  on  a  group  of  twelve  students.  The  pilot  test  asked  students  to  take  the  Kolb  and  Vz-2 
tests,  study  the  material,  and  complete  the  objective  questions  on  the  material  and  subjective  questionnaire.  In  the  pilot- test, 
students  were  given  one  hour  to  complete  the  study  and  were  asked  approximately  how  much  of  the  material  they  were  able 
to  study  thoroughly.  The  one-hour  time  limit  was  found  to  be  adequate  for  the  material  to  be  covered  in  significani  detail. 

Measnregncnt  of  learning  outcome 

Learning  outcome  was  measured  by  a  20  question,  posttest  questiomiaire  based  on  the  material,  which  resulted  in  a 
single  score.  Operational ized,  this  score  was  the  number  of  correct  answers,  minus  .2  for  each  incorrect  answer.  Questions 
were  developed  from  direct  quotations  in  the  material  and  presented  in  a  multiple  choice  format.  All  questions  given 
included  a  fifth  choice  "1  don't  know"  as  an  option  to  reduce  guessing.  This  was  an  effort  to  collect  an  accurate  measure  of 
the  amount  of  material  learned  and  comprehended  from  reading  the  material.  C^iestions  also  were  included  in  the  pilot- test  to 
both  evaluate  readability  and  to  assess  the  assumption  that  this  population  of  students  had  little  or  no  knowledge  of 
hypertext.  Students  in  the  pilot  group  reported  Uiat  they  understood  the  questions,  and  in  addition  had  no  prior  knowledge 
of  hypertext. 

Reliability  of  Learning  Outcome  Measure.  A  Kuder-Richardson  21  reliability  test  was  nni  on  the  completed 
questionnaire  from  282  actual  subjects.  A  K-R  21  score  of  .65  on  the  aggregate  data  suggests  satisfactory  reliability.  If  the 
questionnaire  were  to  be  used  for  a  second  or  follow  up  analysis,  a  K-R  20  and  individual  item  cuialysis  would  be  calculated 
to  revise  the  survey  and  improve  its  overall  alpha  value. 

Measurement  of  satisfaction  levd 

A  questionnaire  collected  (a)  demographic  information  about  die  student  and  his/her  prior  computer  exi)erience  and  (b) 
the  satisfaction  level  from  having  used  die  method  of  presentation  of  that  material.  (Questions  concerning  user  satisfaction 
measured  satisfaction  (or  dissatisfaction)  with  various  aspects  of  die  text  presentation.  Satisfaction  level  was  measured  with 
a  questionnaire  of  12  questions  with  five  point  Likert  scale  responses.  Those  12  questions  asked  for  responses  in  four 
general  areas;  general  satisfaction,  satisfaction  with  die  text  structure,  disorientation  effects  caused  by  hypertext  links,  and 
satisfaction  with  die  material  itself. 

Factor  Analysis  on  Satisfaction  Variables.  A  factor  andysis  pcrfoniied  on  die  12  satisfaction  variables  produced  four 
factor  scores,  each  relating  to  the  four  general  areas  of  satisfaction.  Therefore,  when  analyzing  satisfaction  levels,  die  analysis 
considered  dicsc  four  factor  scores  instead  of  the  actual  12  questions  concerning  satisfaction.  (See  Tabic  3  and  l  ablc  4) 
Using  factor  scores  in  place  of  individual  questions,  provides  a  way  of  summarizing  a  composite  view  of  the  user  satisfaction 
dimension.  These  factor  scores  provide  independent  measures  of  satisfaction  rather  than  using  sets  of  quesdons  diat  would 
not  necessarily  be  inde^Tcndent. 


Insert  Table  3  and  Table  4  about  here. 
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As  Table  3  shows,  four  faclore  had  eigenvalues  greater  lhan  1 .0,  cumulatively  explaining  two  thirds  of  the  variability. 
The  factor  matrix  (See  Table  4)  identifies  the  four  factors  which  were  converged  in  6  iterations  by  a  Varimax  rotation. 
Factor  1,  (questions  1, 5, 8, 10,  and  12)  arc  those  questions  asking  about  general  satisfaction  with  the  the  lesson.  Factor  2 
(questions  3, 9,  and  1 1)  were  those  reflecting  tlie  text  structure  and  tlius  the  ability  to  choose  the  order  by  which  to  view  the 
material.  Factor  3  (questions  2  and  4)  reflected  the  amount  of  disorientation  caused  by  the  hypertext  movement.  The  fourth 
factor,  (questions  6  and  7)  measured  the  extent  to  which  the  student  enjoyed  reading  about  the  material  in  the  lesson. 

Jakob  Nielsen  (1990a)  supports  the  use  of  user  satisfaction  as  one  measur^iient  of  softwaie  usability.  Satisfaction  was 
used  in  his  research  to  determine  whether  a  particular  method  of  a  hypertext  lesson  was  preferred  by  a  particular  learning 
style,  aad  would  thus  have  a  greater  probability  of  being  used.  This  was  based  on  the  assumption  that  a  user  will  use  a 
program  if  he  likes  that  method  of  presentation  and/or  navigalicm  techniques. 

Reliability  of  Satisfaction  Measures.  An  aLalysis  of  282  student  responses  to  the  12  satisfaction  questions  produced  a 
.85  Cronbach  alpha  value.  Thus,  the  use  of  tliese  satisfaction  questions  and  the  resulting  four  factor  scores  is  considered  an 
acceptable  reliability  level. 

Opergttonal  view  of  the  variables; 

The  independent  variables  are  Cognitive  Style  as  measured  by  the  LSI  and  Vz-2  and  the  Treatment  Method. 

LSI.  The  LSI  is  measured  with  an  inler/al  scale  diiat  determines  a  nominal  assignment  to  one  of  four  quadrants  and 
identify  the  subject's  learning  style. 

VZ"!.  The  Vz-2  is  measured  with  an  interval  score.  Often  referred  to  as  the  Paper  Folding  Test,  the  Vz-2  presents  the 
subject  with  drawings  illustrating  a  series  of  ^ wo  to  three  folds  in  a  square- piece  of  paper  and  a  hole  punched  in  the  paper 
after  the  last  fold,  'fiie  subject  then  is  asked  to  select  one  of  5  drawings  that  shows  the  correct  version  of  how  the  paper 
would  look  when  imfolded.  The  subject  has  three  minutes  to  complete  the  first  10  drawings,  and  another  three  itiinutes  for  a 
second  set  of  10  drawings.  Tne  score  of  the  Vz-2  is  calculated  by  adding  one  point  for  each  right  selection,  and  subtracting 
1/5  point  for  each  wrong  selection.  This  results  in  a  maximmn  possible  score  of  20  and  a  minimum  of  -4  (Ekstrom,  1976). 

Treatment  Mediods.  (See  Appendix  A)  The  treatment  method  also  will  be  a  noaniual  scale  widi  four  methods 
identified.  All  four  will'differ  only  in  the  presentation  method  of  the  material.  The  four  text  treatment  groups  are:  (1) 
Linear/sequential  text  only,  with  only  a  page  forward,  page  back,  or  return  to  the  table  of  contents  available.  (2)  A  hypertext 
version  of  the  report  with  buttons  available  to  move  to  other  parts  of  the  report,  but  without  any  visual  map  of  the  structure 
of  the  information  available.  (3)  a  hypertext  version  of  the  report,  with  a  sequentially/vertically  organized  interactive  map 
available  at  all  times  that  will  not  only  serve  as  a  map,  but  also  as  a  menu  to  select  other  parts  of  the  report,  and  (4)  a 
hypertext  version  of  the  report,  with  a  spatially  organized  interactive  map  available  at  all  times  that  will  serve  as  a  map  and 
as  a  menu  to  select  other  parts  of  the  report.  All  of  ih*.  methods  will  be  in  the  fonn  of  a  HyperCard  stack  with  identical  text 
material,  differing  only  in  navigation  options. 

The  dependent  variables  ar<;  boti*  x)nstructed  and  interpreted  using  interval  scales.  There  arc  two  separate  categories  of 
the  dependent  variable  measures.  Learning  outcome  is  measured  by  taking  a  posttest  of  twenty  questions  thai  were  based  on 
the  material  presented  in  the  lesson.  Questions  were  constructed  directly  from  each  major  section  in  the  hypertext  dociuncnt. 
The  level  of  satisfaction  measured  the  student's  subjective  response  to  llie  treaUnent  method.  Specifically  12  questions 
measuring  the  subjects  response  to  the  major  issues  in  hypertext  navigation,  such  as  disorientation,  ability  to  keep  a  mental 
map,  and  whether  they  prefer  to  follow  a  linear  path,  or  have  associative  paths  to  follow.  A  factor  analysis  identified  four 
dimensions  of  satisfaction  and  these  foiu*  factors  were  used  to  measure  satisfaction.  Both  independent  and  dependent 
variables  are  summarized  in  Table  1 . 

Study  Design:  The  experimental  design  used  to  examine  learning  outcome  and  learner  satisfaction  is  llie  "Posttest  Only 
Design."  (Campbell  and  Stanley,  1963).  In  addition,  for  this  research,  each  treatment  group  serves  as  a  control  for  the  next, 
by  holding  constant  factors  in  die  previous  method  and  adding  one  additional  treatment  variation.  For  example,  the  first 
treatment  method,  linear  presentation  of  the  text  only  without  hypertext  measures  factors  without  associative  linking  effects. 
The  second  treatment  method,  hypertext  without  a  map,  provides  a  method  fcMr  measuring  the  effect  of  associative  hnks.  TTie 
third  method  assesses  the  impact  of  adding  a  navigational  map  and  the  fourth  method  assesses  the  additional  impact  of  a 
spatially  organized  navigational  map. 

The  resulting  study  can  be  demonstrated  with  a  tliree-dimeusional  diagram  (refer  to  Figure  2).  The  analysis  required  a 
4X4  +  covariate  ANO VA  design.  The  main  effects  of  the  ANO VA  are  Treatment  Method  and  Learning  Style,  with  Spatial 
Abihty  serving  as  a  covariate  in  an  analysis  of  covariance. 

Results 

Multiple  ANO  VA  procedures  were  conducted  for  eadi  of  the  dependent  variables.  For  example,  to  test  tlie  first  null 
hypothesis,  ''There  are  no  significant  differences  in  learning  outcome  or  satisfaction  level  among  different  learning  style 
groups  of  students.^'  an  ANOVA  procedure  was  used  to  indicate  the  amount  of  variance  of  a  student's  learning  outcome 
between  cognitive  styles,  and  separate  ANOVA  procedures  were  used  for  each  of  the  satisfaction  factors.  The  sums  of 
squares  fw  each  of  the  independent  variables  (Method,  and  LSI)  were  the  main  effects,  while  the  interactions  between  them 
were  tested  separately.  Hypotheses  were  tested  by  the  ratio  of  the  mean  squares  of  each  source  of  variation  variable  to  the 
mean  square  for  llie  residucnl.  In  addition,  the  use  of  a  covariate  (a  student's  spatial  ability)  was  added  to  adjust  the 
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measurement  of  the  main  effects  to  account  for  that  aspect  of  cognitive  style.  For  this  research,  a  significance  level  of  .05 
was  used.  The  data  were  analyzed  to  address  the  four  hypotheses  given  in  the  preceding  section.  Each  of  the  hypotheses 
were  addressed  in  five  stages,  first  as  ihcy  relate  to  learning  outcome,  and  then  in  regard  to  satisfaction  levels  measiu-ed  by 
the  four  satisfaction  factor  scores. 

Learning  Outcome  agid  Sfttisfaction  Factor  Score  Means 

The  mean  scores  for  learning  outcome  and  the  four  satisfaction  factor  scores  are  presented  in  Tables  5  and  6.  These 
tables  will  be  used  in  the  analysis  of  the  data  reported  in  this  chapter  and  inlerpreling  these  results  in  the  following  chapter. 

Insert  Tables  5-12  about  here 


Learning  Outcome  Scores.  The  learning  outcome  score  means  for  each  of  the  study  design  groups  are  presented  in 
Table  5  along  with  the  number  of  subjects  in  each  group  and  the  standard  deviation  for  each  group.  As  indicated  in  the 
iable,  the  mean  score  of  divergcrs     =  8.25)  was  the  highest  of  the  four  LSI  types.  Presentation  Method  1  had  the  highest 
overall  mean  score  (jt  =  8.81 ). 

Satisfaction  Factor  Scores.  Tlie  factor  scores  generated  in  the  factor  analysis  of  satisfaction  questions  for  each  of  the 
study  design  groups  are  presented  in  Table  6.  Data  in  ilsis  table  will  be  used  tliroughout  this  chapter  and  die  next  two  in  the 
analysis  of  learner  satisfaction  means. 

Sumymarv  of  Rfsults 

The  initial  results  of  the  statistical  analysis  are  summarized  for  of  each  of  the  four  hypotheses,  witli  the  four  dependent 
variables  (learning  outcome,  general  satisfaction,  satisfaction  with  text  structure,  disorientation  effects,  and  satisfaction  with 
the  material  content)  listed  for  each  hypothesis  Tlicse  results  are  presented  in  Table  7.  As  described  in  the  previous  chapter, 
the  method  of  analysis  first  used  an  ANOVA  procedure,  matching  the  study  design  in  Figure  2.  Mediod  of  presentation  and 
learning  style  were  used  as  the  main  effects  on  learning  outcome  wiUi  spatial  ability  (Vz-2)  used  as  a  covariate.  The 
remainder  of  this  chapter  will  address  the  statistical  results  for  each  of  the  cicpendenl  variables  for  cacli  hypothesis. 

H  9  4i  Learning  Style  Differences 

Learning  Outcome  among  I^caming  Styles.  As  shown  in  Table  8,  differences  among  the  mean  scores  between  learning 
styles  (F3,265  =  .899,p  =.442)  were  not  significandy  different.  Thus  Uie  first  part  of  HI0,  diat  Uiere  would  be  no  significant 
differences  of  learning  outcome  scores  among  learning  styles,  can  not  be  rejected,  and  the  idicniatc  hypothesis  can  not  be 
supported. 

Satisfaction  gmong  learning  Styles.  As  described  in  die  previous  chapter,  learner  satisfaction  was  measured  wiUi  four 
factor  scores.  Thus,  to  lest  the  second  part  of  the  first  three  hypotheses,  four  separate  ANOVA  procedures  were  used  for  each 
of  the  satisfaction  factors. 

General  satisfaction:  as  shown  in  Table  9,  differences  in  general  satisfaction  between  learning  styles  were  not  significant. 
(F3.265  =  .285,/?  =.836)  Satisfaction  with  Text  Structure;  Table  10,  indicates  differences  in  satisfaction  widi  the  structure 
of  the  text,  between  learning  styles  (F3  255  =  .778,p  =.507)  were  not  significandy  different.     Disorientation  Effect: 
differences  in  satisfaction  levels  between  learning  styles  as  affected  by  die  amoiml  of  disorientation  were  not  significantly 
different,  F3  265  =  584,/?  =.626.  (See  Table  11)  Satisfaction  with  Material  Content:  as  shown  in  Table  12  differences  in 
satisfaction  widi  material  content  between  learning  styles  were  not  significandy  different.  (F3  265=  195,/?  =.900) 

As  indicated  by  the  four  ANOVA  tables,  die  second  part  of  Hl^j,  *UJ?ere  would  be  no  significant  differences  of 
satisfaction  levels  among  learning  styles",  could  not  be  rejected,  and  the  allcmate  hypothesis  ciin  not  be  supported. 

H2^  Spatial  Ability  Differences 

Learning  Outcome  and  Spatial  Ability.  As  shown  in  Tabic  8  differences  among  die  mean  learning  outcome  scores  based 
on  spatial  ability  (die  covariate  Vz-2)  were  not  significantly  different.  (F|  255  =  559,/?  =.455)  Thus  die  first  part  of  H20, 
diat  there  would  be  no  significant  differences  of  learning  outcome  scores  among  different  spatial  abilities*',  can  not  be 
rejected,  and  die  alternate  hypothesis  can  not  be  supported. 

Satisfaction  and  Spatial  Ability.  As  in  die  previous  section,  learner  satisfaction  based  on  differences  in  s|)aiial  abilides, 
were  analyzed  using  the  four  sc|>arate  ANOVA  procedures  for  each  of  die  satisfaction  factors. 

General  satisfaction;  as  shown  in  Table  9,  general  satisfaction  between  different  spatial  abilities  (the  covariate  in  the 
ANOVA)  indicated  sigmficanl  differences.  (Fi  ies  =  6.941 ,  p  =.009)  The  conelalion  between  general  satisfaction  and  Vz-2 
is  quite  significant  (p  =  .005)  and  is  also  a  negative  correlation.  Satisfaction  with  Tcxi  Structure:  Table  10  indicates 
differences  in  satisfaction  with  the  structure  of  text,  between  different  spatial  abilities  (1^265  =  2.775,/?  =.097)  were  not 
significantly  different  at  the  .95  significance  level.  However,  with  a  90%  significance  level,  it  would  be  of  some  benefit  to 
look  at  die  differences  that  were  shown,  llie  correlation  coefficient  indicated  a  slighQy  negative  correlation  between  Vz-2 
and  die  second  factor  score  at  a  significance  level  of  .052.  Disorientation  Effect;  differences  in  disorientation  levels  between 
different  spatial  abilities  were  significandy  different.  (Sec  Table  4.7)  (F,  255  =  7.007,  p  =.009)  A  correlation  coefficient  of 
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-.1542  at p=.O05  indicales  a  sigiuficaiil  difference  aii>ong  spalial  abilities.  However  the  faet  lliat  ilie  correlation  is  so  small 
and  in  the  opposite  direction  tlian  one  would  expect,  suggests  thai  the  coirelation  is  not  one  which  would  affect  differences 
in  learning  with  a  hypertext  documenl.  Satisfaction  wiSi  Material  Content;  as  indicated  in  Table  12,  differences  in 
satisfaction  with  material  content  between  spalial  abilities  were  not  significantly  different.  (Fs  z^s  =  2.405,  p  =.122) 

Therefore,  the  second  piirt  of  1 120,  "ihere  would  oe  no  significant  differences  of  satisfacHon  levels  among  spatial 
abilities   produced  mixed  results,  with  at  leas;  some  sup|X5ri  for  differences  in  satisfaction  levels  by  spatial  ability,  (jcncral 
satisfaction,  and  disorientation  indicate  significant  differences  among  differing  spatial  abilities.  Satisfaction  with  the 
flexibility  of  hypertext  is  supjxirled  at  the  90%  confidence  level,  while  f.^jatial  ability  appe<irs  to  have  little  effect  oh* 
satisfaction  with  the  matcricil  content. 

PresentaHon  Method  Differences 

Learning  outcome  scores  and  Presentation  Method.  1  ne  differences  among  these  means  were  significant 
(^3.265  =  2.983,  p=. 032),  (Sec  Table  8)  although  aecoimling  for  a  very  small  portion  (3%)  of  the  difference  among  learning 
types.  A  multiple  range  test  using  a  Duncan  Procedure  identifies  tJie  differences  between  Presentation  Mctiiod  1  and  the 
other  three  as  being  significant  at  the  .05  level.  Tlie  mean  scores  of  those  subjects  who  were  presentc<I  witli  Method  I  (j4  = 
8.81),  were  liigher  than  for  tliose  who  were  presented  with  hy|>crtcxt,  with  or  without  a  graphical  map  (/^  =  7.51 , 7.30,  and 
7.38  for  Methods  2, 3  and  4  rcsjKctivcly).  These  results  indicate  that  the  first  part  of  HB^,  stating  'Uliere  would  be  no 
significant  differences  of  learning  outcome  scores  among  presentation  methods",  can  be  rejected,  and  an  alternate  hy|X)llicsis 
supported. 

Satisfaction  Level  and  Presentation  Metliod.  To  analyze  the  relationship  between  learner  satisfaction  levels  and  the 
method  of  presentation  of  hypertext  material,  four  separate  factor  scores  were  studied. 

General  satisfaction;  An  analysis  of  variance  of  general  satisfaction  (Table  9)  (rs.zes  =  .538,p=.657)  indicates  that 
presentation  method  did  not  account  for  a  significant  difference  in  general  satisfaction  level. 

Satisfaction  with  text  structure  There  is  support  of  a  significant  difference  (F3  265  =  5.883, /jcOO  1 )  in  the  satisfaction 
with  text  structure  among  presentation  methods.  O^l^lc  10)  A  Dunaui  Procedure  for  a  multiple  range  test  identifies  a 
significant  difference  at  3ic  .05  level  between  a  text  only  presentation  melJiod  and  those  providing  hypertext.  Tlie  text  only 
presentation  method  was  the  only  mcthcxl  with  a  negative  meati  sairc  for  the  second  factor  sairc  relating  to  hypertext.  This 
result  however  is  in  keeping  with  cxjKXtalions  since  the  text  only  presentation  did  not  allow  hypertext  jiunps.  Figure  5 
indicates  the  differences  between  j)rcsentation  methcxls  and  gencnil  satisfaction.  Tliere  is  a  clciir  distinction  of  satisfaction 
wiUi  a  lesson  presented  with  a  liy|>crtext  fomiat.  This  difference,  however,  is  a  factor  of  the  interaction  between  nietluKl  of 
presentation  and  learning  style  which  is  discussed  in  the  next  section. 

Disorientation  effect  on  satisfaction.  The  amount  of  disorientation  increased  with  the  iunount  of  hy|>crtext  provitlctl. 
(Sec  Figure  5)  These  differences  in  the  imioimt  of  disorientation  were  significant  (I'3  265  =  4,819,p=.0()3)  and  account  for 
about  5%  of  the  variance.  CFablc  1 1)  A  multiple  range  test  using  the  Duncan  procedure  indiaites  a  significant  difference 
between  a  textual  presentation  method  and  those  with  hypertext  links.  In  addition,  the  means  for  tlie  disorientation  effect  by 
presentation  melliod  indicate  llic  increasing  amount  of  the  eifect  as  the  amount  of  flexibility  is  provided. 

Satisfaction  with  Material  Content.  Table  12  indicates  a  significant  difference  (F3.26S  =  5.530,/.^.O01)  in  the 
satisfaction  with  the  actual  material  being  presented.  As  Figure  6  illustrates,  students  were  dissatisfied  with  the  linear  text 
only  presentation,  and  more  satisfied  with  a  hypertext  map  using  a  spatial  navigation  map.  A  multiple  range  test  using  the 
Dmican  procedure  shows  the  significant  differences  between  a  textual  presentation  method  and  both  versions  of  navigation 
methods  in  a  hypertext  lesson,  and  aJso  a  difference  between  a  simple  hypertext  lesson  and  a  spatial  navigation  map.  The 
means  of  these  methods  indicate  a  positive  satisfaction  with  hyiK^rtext  with  either  form  of  a  map  iuid  a  negative  satisfaction 
level  for  textual  presentation  only. 

ITius,  the  second  part  of  H30,  "tliere  would  he  no  significant  differences  of  satisfaction  levels  among  presentation 
methods'^  can  be  rejected  and  an  alternative  hypothesis  supported.  Although  there  was  not  a  significant  level  of  difference  in 
general  satisfaction,  there  clearly  were  significant  differences  between  textual  presentations  and  liypcrlcxt  based  methods. 

H40  Interaction  effects 

Interaction  effects  between  Learning  Styles  and  IVcscntation  Mctliod.  There  iirc  two  aspects  of  interaction  effects  that 
we  are  interested  in,  the  affect  on  leaniing  outcome  and  on  satisfaction  levels.  As  indicated  in  Table  8  there  was  no  sup|X)rt 
for  a  significant  difference  in  leaniing  outcomes  and  cuiy  interaction  effects  between  learning  style  and  presentation  methods. 

The  second  aspect  of  1 14^,  ^'There  are  no  significant  interaction  effects  between  learning  style  and  treatment  method  in 
their  effect  on  satisfaction  level"  requires  an  analysis  of  several  as|>ccts.  Analysis  of  vaiiaiicc  of  the  general  satisfaction 
level  (F9265  =  2.412,p=.012)  indicates  a  significant  difference  in  the  interaction  effects  between  learning  style  and 
presentation  method  accounting  for  7%  of  the  variance.  (See  Table  9)  At  tlie  same  time,  an  iuuUysis  of  the  oUier  three 
satisfaction  factor  scores  do  not  indicate  a  significant  difference  in  the  interaction  effect. 

Figure  3  illustrates  a  wide  difference  between  general  satisfaction  with  hypertext  lessons  between  students  who  prefer 
active  experimentation  (convergers  and  accoinodators)  and  those  who  prefer  reflective  obser\  ation  (assimilators  and 
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divergers).  As  the  line  chiirl  shows,  there  does  appear  graphically  to  be  a  distinetion  between  these  two  groups  in  their 
preference  of  presentation  methods  when  hyjiertext  is  involved.  In  addition,  as  multiple  range  tests  using  the  Duncan 
procedure  (See  Tables  10. 1 1 ,  and  12)  indicate,  a  significant  difference  between  the  satisfaction  levels  with  a  text  only 
presentation  method  and  the  use  of  hypertext.  In  all  three  cases,  students  indicated  a  higher  level  of  satisfaction  with  sonic 
form  of  the  use  of  hypertext. 


Summary  ofresuics 

To  summarize  the  results  of  the  statistical  analysis,  Table  13  lists  Uie p  values  for  each  of  the  dependent  variables,  for 
each  hypothesis.  Tlie  results  for  each  of  the  hyjwthcsis  are  next  sununarizcd  according  to  th<;  statistical  hyjx>thcses. 

Hl0.  There  are  no  sigmficmt  differences  in  outcome  measures  or  satisfaction  level  among  different  learning  style  groups 
of  students. 

There  was  no  support  for  a  signifieant  difference  in  the  learning  outcome  or  satisfaction  levels  based  on  Rolb'i;  Ixxirning 
Style  Inventor}'.  Thus  the  null  hypothesis  can  not  be  rejected. 

1120.  There  are  no  significant  differences  in  the  means  of  learning  outcome  or  satisfaction  level  among  different  spatial 
abilities  of  students. 

There  was  no  statistical  siipp')rt  for  differences  in  learning  outcome  based  on  a  students  spatial  ability.  Satisfaction 
levels  however  did  have  some  st.  » ^lical  support  for  a  significant  level  of  difference  basetl  on  spatial  <ibility.  Thus,  tlie  null 
hypothesis  relating  to  Icaniing       ome  can  not  be  rejected,  llie  null  hypotliesis  relating  to  satisfaction  levels  based  on 
spatial  ability  however  can  be  rejected  and  conclusions  made  about  tlie  spatial  ability  and  its  affect  on  satisfaction  with  a 
learning  system. 

H30.  There  are  no  significant  differences  in  the  means  of  learning  outcome  or  satisfaction  level  among  different  treatment 
method  groups  of  students. 

There  were  statistically  significant  differences  in  both  the  learning  outcome  and  satisfaction  levels  based  on  presentation 
methods.  The  null  hy{)othesis  can  be  rejected  and  conclusions  made  about  the  impact  of  method  of  presentation  on  Icaniing 
with  a  hypertext  assisted  lesson. 

H40.  There  are  no  significant  interaction  effects  between  learning  style  and  treatment  method  in  their  effect  on  learning 
outcome  and  satisfaction  level. 

Tliere  was  no  support  for  statistical  differences  in  Icaniing  outcome  based  on  the  interaction  effects  between  lc<uTiing 
style  and  methods  of  presentation.  There  was  statistical  supi)ort  for  significant  differences  in  the  interaction  effects  between 
learning  style  and  methods  of  presentation  on  1  earner  satisfaction  levels. 

Discussion  of  Findings 

This  study  investigated  the  interactions  between  cognitive  stylets  (measured  by  learning  styles  and  spatial  ability)  and 
navigation  maps  in  hy|x:rtext  assisted  Icaniing.  Learning  outcome  and  user  satisfaction  were  used  as  dependent  viuiables  in 
ail  analysis  of  the  interaction  of  learning  styles  and  mental  maps  in  the  design  of  hypertext  documents  used  for  exploratory 
learning. 

Three  research  questions  were  addressed:  (1)  Does  coiinitivc  style  (learning  style  or  spatial  ability)  affect  lauiiing 
outcome  or  user  satisfaction  in  hypertext  assisted  leaniiiig?  (2)  Does  llie  presentation  method  of  navigational  maps  affect 
learning  outcome  or  user  satisfaction  in  hypertext  assi&ted  learning?  (3)  Are  there  interaction  effects  between  cognitive 
styles  and  method  of  presentation  wliich  affect  learning  outcome  or  user  satisfaction  in  hypertext  assisted  Icaniing?  This 
section  will  address  each  of  the  three  general  questions. 

Does  cognitive  style,  cither  learning  style  or  spatial  ability,  affect  luuiiing  outcome  or  user  satisfaction  in  liy|)ertcxt 
assisted  learning?  This  question  was  addressed  in  two  parts,  first  the  effect  of  learning  style  iuid  spatial  ability  on  Icaniing 
outcome,  and  then  the  level  of  satisfaction.  The  lack  of  suj^jwrt  for  learning  style  having  cuiy  effect  on  leanung  outcome  or 
user  satisfaction  was  rather  uncxixjcted.  As  indicted  in  the  literature  review,  there  has  been  much  research  on  the  affects  of 
learning  styles  o/i  various  Icaniing  environments.  In  addition,  there  has  been  support  for  Kolb's  le^u-ning  style  inventory  as 
an  indicator  of  differences  in  Icaniing  in  previous  research,  i  lowevcr,  as  Hiiber  proposes,  there  are  more  variables  involved  in 
cognitive  styles  than  we  currently  know,  and  therefore  any  |X)sitive  results  in  the  area  may  be  of  limited  use.  The  definition 
of  Kolb's  learning  styles  as  a  ineasmement  of  a  student's  'preferred'  learning  modes,  would  iiidic<itc  tha:  the  LSI  would  be  a 
valid  measurement  of  the  iuiiouiit  of  satisfaction  from  different  Icaniing.  1  lowevcr,  one  oQier  factor  which  may  affect  l«miing 
outcome  iuid  satisfaction  is  the  aiiiount  of  time  necessary  (o  Icani  '?iow-to-lcani'  in  a  liyiKTtexl  environment,  lii  addition,  by 
its  nature  hypertext's  strengths  and  weaknesses  do  not  become  apparent  until  the  si/x:  of  a  d(H:uiiicnt  Ix^comes  lcU*ger  th«ui  one 
caii  m<'iiiitain  in  a  mental  map. 

Spatial  ability,  while  not  having  a  signifiuuil  effect  on  Iciiniing  outcome,  did  provide  sigiiifiuuit  differences  among 
lc<inicrs  IVevioiis  stiuhcs  using  the  PajKr  I'bldtng  Test  liav  e  also  found  it  to  be  quite  accurate  in  measuring  differences  in 
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the  use  of  compulcrs.  Il  is  not  surprising  thai  a  person's  spatial  ability  affects  satisfaction  with  a  specific  interface.  What  is 
more  difficiiU  is  to  identify  what  types  of  presentation  methods  are  best  matched  to  different  individiKils. 

Does  the  presentation  :net]iod  of  navigational  maps  affect  learning  outcome  or  user  satisfaction  in  hyixirtext  assisted 
learning?  Perhaps  the  most  helpful  findings  of  tliis  study  are  the  importance  of  ilie  design  of  tlic  presentation  method  of  a 
learning  environment.  Tiie  response  of  this  study  to  this  particular  question  provides  two  different  results.  First,  wliilc  not 
significandy  differcDl,  learners  had  a  higher  learning  outcome  with  a  text  only  document.  At  ihc  smne  time,  students  re|>ortcd 
a  significaiU  level  of  diifTcrcncc  in  their  satisfaction  level  among  tlie  four  learning  nietliods.  'I'lieie  was  a  detinitc  separation 
of  active  ex|}cnmentation  learners  from  reflective  observauon  ieanicrs  in  the  level  of  satisfaction.  While  those  who  preferred 
reflective  observation  were  less  satisfied  with  the  use  of  hypertext  links,  ihey  were  more  satisfied  with  liypertcxt  documents 
when  provided  with  a  navigation  map.  This  indicates  that  when  care  is  taken  to  provide  assistance  in  a  hypertext  learning 
process,  differences  among  different  learning  styles  are  reduced.  Therefore,  this  research  concludes  that  while  learning 
outcome  was  higher  for  non-hypertext  methods,  tlic  satisfaction  with  hypenext  indicates  the  ix)tenti*»l  of  such  systems, 
provided  an  effective  interface  design. 

Are  there  interaction  effects  between  cognitive  styles  and  method  of  presentation  which  jiiTcct  Iciiniing  outcome  or  user 
satisfaction  in  hypertext  assisted  learning?  Learning  outcome  wa.s  not  affected  by  interactions  between  cognitive  styles  and 
methods  of  presentation.  In  addition,  specific  aspects  of  satisfaction  (preference  for  hypertext,  mid  disorientation)  also  lacked 
support  for  interaction  effects.  1  lowever,  a  student's  general  level  of  satisfaction  with  a  metluxl  of  a  hyjHirtext  learning 
environment  did  have  a  significaiU  difference  in  the  amoimt  of  interaction  effects.  Ihis  would  iniply  that  there  is  some 
support  for  further  research  into  what  specific  cognitive  design  issues  <\ic  important  for  future  izUeriacc  ilcsign  of  hy|)ertext 
navigation  maps. 

Limitations  of  the  study 

In  Older  to  accurately  measure  asjKcts  of  hypertext  learning  such  as  disorientation  and  differences  in  learning  outcome, 
the  hypertext  documctit  must  be  of  a  large  enough  nature  to  cause  the  cognitive  overhead  to  require  sjKcille  assistance  from 
the  learning  system.  Such  was  not  the  case  of  this  study.  Instead,  as  widi  much  research,  the  need  for  controlling  time  and 
effort  spent  among  students,  required  that  the  size  of  Ihe  material  be  kept  to  a  manageable  si/.e.  The  use  of  a  hu-gc  document 
(more  than  1  ,(XX)  nodes  of  infonnatioii)  might  have  created  a  more  signincaiit  level  of  difference  in  both  learning  outcome 
and  satisfaction.     Another  limitation  was  that  the  level  of  technical  detail  and  nature  of  the  material  included  in  the  lesson 
may  not  have  been  appropriate  for  aii  introductory  computer  class.  In  addition,  satisfaction  with  the  lesson  presentation  was 
measured  in  general  terms  for  each  of  the  areas  of  interest.  For  example,  ilic  satisfaction  questions  dealing  with 
disorientation  only  measured  the  general  amount  of  disorientation.  Further  details  about  siKcific  asjKcts  of  what  caused 
disorientation  would  help  in  the  analysis  of  the  results. 

In  hindsight,  a  larger  hypertext  docunieut,  with  simpler  concepts  of  specific  interest  to  this  population  of  students  may 
have  produced  significandy  different  results.  More  specific  information  on  satisfaction  viuiables,  and  what  le\'cl  of  learning 
was  achieved,  would  greatly  enhance  the  findings  of  a  study  such  as  this. 

Conclusions  and  Implications 

Conclusions  from  this  study  indicate  that  while  dicre  are  dilTerences  in  some  asi>ects  of  cognitive  style,  such  as  spatial 
ability,  it  could  not  be  dclennined  that  learning  style  affects  either  learning  outcome  or  satisfaction  in  the  use  of  a  hypertext 
lesson.  However,  the  design  ol'  navigation  methods  for  presenting  hyiHirtcxt  material,  docs  sigiiific<uitly  <iffcct  botli  Iciu-ning 
outcome  ajid  the  level  of  satisfaction  in  using  such  a  method. 

Implications  for  the  use  of  hy|)crtext  in  different  applications  ixtc  evident.  I'irst,  tlierc  aie  some  dilTcrcnccs  in  cognitive 
style  which  affect  the  success  of  such  a  system.  Therefore,  while  this  study,  and  many  oilicrs  to-date,  have  not  definiUvely 
identified  wliich  of  those  cognitive  style  as;)ects  hold  the  most  ix)tcntiiil  for  inii)roviiig  le<u"niiig,  iherc  is  support  of  the  need 
for  future  research,  Specifiailly,  user  satisfaction  has  been  shown  to  In;  a  successful  indiaitor  of  lc»u  niiig  success  in  other 
environments.  Clearly,  satisfaction  is  alTccicd  by  the  interface  iiieihiKl  provided  with  a  learning  system.  Therefore,  when 
contemplating  the  design  of  the  user  interface  of  both  hy|)crtexl  dtKunients  and  learning  environments,  more  information  on 
specific,  critical  issues  that  have  a  significant  affect  on  the  success  of  that  system  would  be  of  interest  to  designers. 

Secondly,  the  design  of  the  presentation  method  of  the  materiid  did  have  a  statistically  sigiiifiauit  jilTcct  on  the  leiiniing 
outcome  of  a  lesson.  Therefore  future  research  should  continue  to  ideiitif>  iliosc  aspects  of  a  user  interface  wliich  would 
improve  the  learning  outcome  for  students. 
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Appendix  A 
Presentation  Methods 


TTw  Fai»»r  of  Hci[wt»xl  (TH  Htlioni 
Tb»  cowr  of  the  vwt^  vord  'hypertext *,  Ttd  Nthon  from  Xwdu  9*y«  « 
so(n«wh«t  r«mb1irtg  i»Tk  on  scm  of  hts  HT  -MJe^s.  K       inustratrd  with  *  cotor'ful 
eolkotlon  of  slVlts  from  pr^v^am  t«1ks  -  soctm  9oir>9  btcfc  to  1 970.  Kt  vinttd  to 
s««  compottrs  uMd  for  thf  <]M>«rAH^^d  hindlirvg  of  M^as  «nd  Mid  that  th*'  v^y 
personal  oomputx>9  is  ourrtntk^  happ^ftkig  ts  pUin  vrong.  Ptoplf  «r»  drowning  in 
mall  flWj  vlih  tnoom|^«h*<u(fbk  mttms  «r>d  4ncomp«it>1«  softvvt. 

Vh«i  vntir>g  vtth  ourrrnt  computers ,  v*  rwwj  to  do  '<Jou6!»  moint€JV»r«f  *  to  keep 
the  printouts  *t¥S  {{[9  consistent  vhen  nvaJong  ch^n^es  to  cr»  or  (h*  other.  Tl^ 
Mtolntosh  is  just  vh«t  NelscA  e«ns  *  p«p«r  sim<Jl«tor  4nd  not  an  online  solutio^k. 
InstrM)  ve  ne*d  to  bufld  4  world  where  'trt  wn  sh*re  p4()H— tess  viforrnttion  in  thr 
same  v«y  w«  oan  share  paptr :  Books  -are  always  oompatiblel 

r^elsoo  related  how  he  beoame  interested  In  HT:  As  a  student  h«  became  a  serloos 
note-taker  4nd  took  notes  an  the  time.  He  wanted  to  be  At)V  to  £e«  lf«  orfcgtnal 
context  of  the  note  as  well  as  its  current  use  side  by  side^  so  he  started  HT  as  a 
student  project  arid  has  btt n  working  on  it  ever  since,  ki  1 967  he  qtv*  his  system 
the  name  X«n«du  and  by  now  ft  U  actuMly  vorkin?  (we  saw  a  demo  at  t^e 
conference). 


5^ 


The  Fath^^f  HgpertrKt  (Ted  Nelson) 

The  oomer  of  the  \ 
somewhat  rambling  talk  on  s 
colWctloft  of  slides  from  previous  talks  -  some  ^oing  back  to  1^ 
see  computers  used  for  the  generaHzed  handling  of  ideas  and  sa 
p«rsona1  computing  is  ourrentk^  happening  ts  pUm  wrong.  Peop 
sn^all  fiWs  vtth  InccmprehensfcTe  names  and  ktcompatf^le  soft^ 


f  very  word  '^hyperteKth,  Ted  Nelson  fromj^ 
!hg  talk  on  some  of  his  HT  ideas,  tt  was  iriosG% 


When  writing  with  our  rent  computers,  we  need  to  do  *  double  n 
th>  printouti  *ftd  file  consistent  when  rmking  ctMn^es  lo  one  or 
\  Mf  cv\tosh]is  just  what  Nelson  catis  a  pap«r  simulator  and  not « 
Instead  we  need  to  bufld  a  worU  where  we  can  stiare  paper 'les 
same  way  we  can  share  p)(>tr ;  Books  are  always  compatible! 

Ntbon  related  how  he  became  interested  in  Hit  As  a  student  h< 
note-taker  «od  took  notes  all  the  time.  He  wanted  to  be  abW  to 
context  of  the  note  as  well  as  its  current  use  sWe  by  side«  so  t 
student  project  and  has  bftn  working  on  rt  ever  cxwe.  in  1 967 
tht  name  Xanadu  and  by  now  it  U  Ktually  working  (we  saw  a  c 
conference). 


A-1  Sequential  Text  Only 


A-2  Hypertext  without  a  Nav 


The  Father  of  Hmwtext  (Ted  f^eteon). 
The  coiner  of  the  very  word  Thypertextf,  Ted  Nelson 
trom|X ar%du]g>ve  a  som«whal  rarnbWig  Ulk  on  some  of 
his  HTTJeas.  ft  was  illustrated  with  a  colorful 
conectton  of  siklas  from  prevknis  t»lks  -  some  going 
back  to  1970  He  wanted  to  set  computers  used  for  the 
generalized  handKng  of  ideas  and  said  that  the  way 
personal  computing  H  currently  happening  is  plain 
wrcng.  People  are  drowning  in  small  fiWs  x'ilh 
inoomprehensiblf  nam»  and  inoompattile  software. 

Yh*n  writing  with  current  computers ,  ve  need  to  do 
"double  maintenarc«  "  to  keep  th«  printouts  and  file 
ooftststent  wlien  making  changes  to  one  or  the  other. 
ThetHaointo5i|is  just  vt«t  Nebon  calls  a  paper 
sknumor  andnot  an  online  sohjtfon.  Instead  we  need  to 
bufld  a  world  vhtre  wt  can  share  paper-leis 
information  in  th*  lam*  way  ve  can  share  paper : 
Books  4re  alwa<ys  oompatfcle! 

N*lson  relat«d  how  he  became  Interested  In  HT:  As  a 
'  studtnt  he  bKim*  a  strtous  note-taker  and  took  notes 
I  all  th*tvT>e.H«  wanted  to  be  able  to  see  the  original 
I  ooniext  of  the  nute  as  wen  as  tts  ourrent  use  side  by 


[Time  since  you  were  here  1 7  seconds 


Research  issues 


Literature 


3 


Current  akapter  •vervt*v  1 

jjyptfttM  Pf  oeit 


Hypertext  pea^le 

The  comer  of  the  very  word  ihypertextj',  Ted  Nelson 
frofr<Xwdub*ve  a  some  what  ramfchng  xalc  on  some  of 
his  HT  idtas.  It  was  illustrated  with  a  color^l 
coHection  of  sh<Ws  from  previous  talcs  -  som*  ^oing 
baok  to  1970.  He  w«nted  to  seeoomputers  used  for  the 
generahzed  handing  of  ideas  and  said  that  the  way 
personal  computfaig  is  currently  happening  Is  plain 
wrong.  Peopte  are  drownteg  in  small  files  with 
ino«mpreltensi>}e  names  and  incovr^atible  software. 

Vhen  writing  with  current  con^ters,  we  need  to  do 
'double  maintenance  '  to  ktfp  the  printouts  and  fHe 
Qonststent  when  making  o^«^ges  to  one  or  the  other. 
ThefMaomtosh^  just  what  Nelson  calls  a  paper 
simulator  and  not  an  online  solution.  hste«d  we  ne*d  to 
bcfld  a  world  vhare  ve  can  share  paper- less 
information  in  the  samv  way  ve  can  share  piper : 
Books  are  always  compatible! 

Nelson  related  how  he  became  interested  in  HT :  As  a 
rtudent  he  became  a  serious  note'taker  and  took  notes 
all  the  tkne  He  wanted  to  be  able  to  set  the  original 
context  of  the  note  as  well  as  its  otrrent  use  side  by 
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A-3  Hypertext  with  a  textual  Map 


A-4  Hypertext  with  a  Spa; 
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Kolb's  Learning  Style  Inventory 


Concrete 
E>p9(lence 


AcBve 
Expodmentai 
e**ng) 


^^^^  Dvwgef 

Refiecuve 
OOseivaeon 


Abstract 
Ccnceptuaiizalon 
(tinKmg) 


Figure  1  -  Kolb's  Learning  Style  Inventory' 


Study  Design 


uaatntfxa  Stylo 
Dve(Q8f 
Assmitalor 
Conveigef 

Acccmmodalor 


A1 

B-l 

CI 

D1 

A2 

B2 

C2 

D2 

A3 

B3 

C3 

D3 

A4 

BA 

C4 

D4 

<:<««KiAi  Hypenext  Hypertexl  Hypefiexl 
ivAim^y  taxiiiaimap  spatial  map 

Pfesentaton  Meffiod 


Figure  2  -  Study  Design 


SAT  I  -  Overall  SoUsfaclion 


riethodt  hethodZ  hettiod3  MeU»(]4 


Figure  3  -  Overall  Saiisfaclion 


SAT3  -  DIssorientation 


Method  1         Mettxx)2         Method  3       Molted  4 


Figure  5  -  Dissorientation  Effect 


SAT2  -  SallsfoclloD  with  Hypertexl 
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0.4. 

•<>•  AaomaJfttor 

0.2 
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0 

./^                    ,                        -o-  AMimiletor 

-0.2 

-04 

-06. 

y 

-08 

Mclhodl         Mrihod2         Method  3          Method  4 

Figtirc  4  -  Siilislaclion  with  Hypertext 

SAT4  '  Satisfaction  with  Lesson  Material 


t 


MilhodI  M(thod2  Methods        Method  4 


Figure  6  -  Satis fiiclion  with  Material  Content 
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Table  1  List  of  Variables 


Independent  Variables; 

A   Treatmeni  Method 
(Variable  categories) 

1.  Linear  text  only 

2.  Hypertext  without  a  navigation  map 

3.  Hypertext  with  a  textual  navigation  nicip 

4.  Hypertext  with  a  spatial  navigation  map 
B    Individual  Cognitive  Style: 

B 1    Kolb*s  Learning  Style  Inventory 
( VariaWe  categories) 

1.  Diverger 

2.  Assimilator 

3.  Converger 

4.  Accominodator 

B2    Spatial  Ability  /  Paper  f  olding  Test 
(one  quantitative  vanablc) 

Dependcat  Variables: 

A    Objective:  learning  outcome  sc^rc 

(one  quantitative  variable) 
B    Subjective:  satisfaction  level 

(Four  factor  score  variables) 

1.  Factor  Score  1  -  General  Satisfaction 

2.  Factor  Score  2  -  Text  Stnicturc 

3.  Factor  Score  3  -  Disorientation 

4.  Factor  Score  4  -  Content  of  Material 


Table  2  Simiinarv  of  Hypotheses 


Hl0.  Learning  Style  Differences.  There  aie  no 

significant  differences  in  outcome  measiu-es  or 
satisfaction  level  among  different  Icaniing  style 
groups  of  students. 

1 120.  Spatial  Abilities.  There  fire  no  significant 

differences  in  the  nie*'ins  of  leaniing  outcome  or 
satisfaction  level  among  different  S|xitial 
abilities  of  students. 

Presentation  Methods.  There  are  no  significant 
differences  in  the  means  of  learning  outcome  or 
satisfaction  level  among  different  presentation 
treatment  methoti  groups  of  students. 

1 Interaction  Effects.  Tliere  arc  no  significant 

interaction  effects  between  learning  style  and 
treatment  meLhod  in  tiieir  effect  on  Iciiming 
outcome  and  satisfaction  level.  


Tabic  3 

Factor  Analysis  of  Satisfaction  Variables:  Statistics 


  FACTOR  ANALYSIS   

Analysis  Numt>dr  i  LIstwIse  deletion  o1  cases  with  mtssing  values 
Extraction  i  fof  Analysfe  i.  Principal- Componenis  Analysis  (PC) 


tnKlal  Statistics: 

Variable 

Communallty 

Factor 

EJgenvaiue  %Var 

Cum  % 

SI 

1.00C00 

1 

4.46083 

37.2 

37.2 

S2 

1.00000 

2 

1.30219 

10.9 

48.0 

S3 

1.00000 

3 

1.15865 

9.7 

57.7 

S4 

1.00000 

4 

1.07003 

8.9 

66.6 

S5 

1.00000 

5 

.71853 

60 

72.6 

S6 

1.00000 

6 

.66836 

5.7 

78.3 

S7 

1.00000 

7 

.60981 

5.1 

83.4 

S8 

1.00000 

8 

.55446 

4.6 

88.0 

S9 

1.00000 

9 

.44558 

3.7 

91.7 

S10 

1.00000 

10 

.371 1 5 

3.1 

94S 

S11 

1.00000 

11 

.34932 

2.9 

97  7 

S12 

1.00000 

12 

.27108 

2.3 

100.0 

PC  Extracted   4  laclors. 

Flnai  Stailsllcs: 

Variable 

CommunaWy 

Factor 

Eigenvalue  %Var 

Cum% 

SI 

.67302 

1 

4.46083 

37.2 

37  2 

S2 

.79871 

2 

1.30219 

10.9 

48.0 

S3 

.53448 

3 

1.15865 

9.7 

57.7 

S4 

.76741 

4 

1  07003 

8.9 

666 

S5 

.66254 

S6 

.74952 

S7 

.70768 

S8 

.66473 

S9 

.67934 

S10 

.74475 

S11 

.60577 

S12 

.50379 

Table  4 

Factor  Analysis  Of  Satisfaction  Variables;  ?-actor  Matnx 


 FACTOR  ANAL  YSIS  

Varlmax  Rolallon  i.  Extraction  1.  •  Kaiser  Normalizatton 
Varlmax  converged  in    6  Kerallons. 


Related  Factor  Matrix. 

FACTOR  1  FACTOR  2 


FACTOR  3  FACTOR  4 


SB 

.78794 

SlO 

.75740 

S5 

-.74478 

SI 

-.60199 

S12 

44095 

Sif 

76295 

S9 

.71869 

S3 

.64471 

S? 

S4 

S6 

S7 

.86157 
.82130 


•83885 
.80014 


Fac'.or  Transfer  nation  Matrix 

FACTOR  1  FACTOR  2  FACTOR  3  FACTOR  4 


FACTOR  1  62551 

FACTOR  2  -  43406 

FACTOR  3  -.34822 

FACTOR  4  .54688 


V 46930 
•49733 
.51809 
.49480 


.44105 
.54943 
.62760 
.33124 


-.41809 
.51223 

-.46523 
.58864 


220 

21j 


19 


Table  5  Outcome  by 

Learnint^  Style  and  Presentation  Method 


METMOl 

MblHOL 

LSI 

1 

2 

3 

4 

TOTALS 

Dtvergers 

mean 

9.38 

8.67 

7.72 

7.28 

8.25 

8d 

3.12 

2.92 

5.22 

3.19 

3.73 

n 

14 

16 

15 

15 

60 

Assimilators 

fitoan 

6.60 

7.04 

7.55 

7.31 

7.65 

8d 

3.71 

3.36 

3.12 

2.97 

3.30 

n 

22 

19 

20 

22 

83 

Converters 

mean 

8.87 

7.54 

6.71 

8.34 

7.87 

8d 

3.51 

4.96 

2.99 

3.07 

3.72 

n 

19 

18 

19 

19 

75 

Accommodators 

mean 

8.57 

6,88 

7.32 

6.37 

.34 

sd 

2.96 

3.47 

3.12 

3.69 

3.34 

n 

18 

16 

15 

15 

64 

METHOD  mean 

6.81 

7.51 

7.30 

7.38 

7.76 

TOTALS  8d 

3.33 

3.68 

3.61 

3.23 

3.52 

n 

73 

69 

69 

71 

282 

Table  7 

Summary  of  Results 


Summary  of  Results 

HVDOthOSiS 

D  value 

relecl 

Hl0. 

1.  Learning  Outcome 

.442 

no 

2.  QenerafSattefactlon 

.836 

no 

3.  Sattefactlon  wRti  Text  Structure 

.507 

no 

4.  Dteortentation  effects 

.620 

no 

5.  Satlsfaciton  wRh  Material  Content 

.900 

no 

H20. 

Soatta}  AbiHtles  D}f<erenoe8 

1.  Loarnlng  Outcome 

.455 

no 

2.  General  Satisfaction 

.009 

yos 

3.  Satisfaction  wtlh  Text  Structure 

097 

00 

4.  Disorientation  effects 

.009 

yes 

5.  Satlsfactton  wSh  Material  Content 

.122 

no 

H30. 

Presentation  Methods  Differences 

1.  Learning  Outcome 

.032 

yes 

2.  Genera!  Satisfactkm 

.657 

no 

3.  Satisfaction  wNh  Text  Structure 

.001 

yes 

4.  Disorientation  effects 

.003 

yes 

5.  Satisfaction  wKh  Materia)  Content 

.001 

yos 

H4o. 

Interaction  Effects  Dllferences 

1.  Learning  Outcome 

.897 

no 

2.  General  Satlsfadkm 

.012 

yes 

3.  Satisfaction  wRh  Text  Structure 

.574 

no 

4.  Disorientation  effects 

.684 

no 

5.  Satisfaction  wKti  Material  Content 

.164 

no 

Table  6  Satisfaction  Factor  Score  means  by 
Learning  Stvie  and  Presentation  Method 


Satisfaction  Factor  Score  t^^ns 

FACTOR  SCORE  1  (General  Satlsf action) 

Divorger 

Asslmilator 

Converger 

Accomodaior 

METHOD  1 
METHOD  2 
METHOD  3 
METHOD  4 

.1473213 
-.4064046 
.1009928 
1236628 

.0435190 
-.5865575 
.3730503 
.3674708 

.1274950 
.3280063 
-.3016294 
-!  1263760 

.0981950 
.3840330 
-.3877529 
-.1912216 

FACTOR  SCORE  2  (Text  Structure) 

DIverger 

Asslmilator 

Converger 

Accomodator 

METHOD  1 
METHOD  2 
METHOD  3 
METHOD  4 

-.6744029 
-.0462843 
.1124850 
.0554993 

-.4696386 
.2385123 
.2163655 
.0279992 

-.4108353 
.2801728 
.3331 746 

.UOoUDOwT 

-.1015388 
-.2502052 
.2222610 
.4776369 

FACTOR  SCORE  3  (Disorientation) 

DIverger 

Assimilaior 

Converger 

Accomodator 

METHOD  1 
METHOD  2 
METHOD  3 
METHOD  4 

.0615183 
-.0875845 
.1490613 
.4383380 

-.4123635 
.2103351 
.2243373 
.1422626 

-.3954425 
-.0824467 
-.2230592 
.2322715 

-.5232912 
-.1396350 
.1668231 
.4754456 

FACTOR  SCORE  4  (Content  of  Materia]) 

DIverger 

Asslmilator 

Converger 

Accomodator 

METHOD  1 
METHOD  2 
METHOD  3 
METHOD  4 

-.1142075 
.0795037 

-.0233894 
.3818548 

-.3480358 
-.1672814 
.3092674 
.0285498 

-.3933063 
-.2386889 
-.1773449 
.8292379 

-.4200705 
.0764750 
.2334970 
.0793348 

Table  8    ANOVALLearnrng  Outcome  bv 

l-^arning  Style  and  Presentation  Method  with  Spatial  Ability 


-••ANALYSIS  OF  VARIAf^CE*'' 

SCORE 
BY  LSI 

METHOD 
WITH  VZ2 


Sum  ol 

Mean 

SIgnlf 

Source  of  Variation    Squares  DF 

Square 

F 

ofF 

Covariales 

6.879 

1 

6.879 

.559 

.455 

V22 

6.879 

1 

6.879 

.559 

.455 

Main  Effects 

139.478 

6 

23.246 

1.888 

.083 

LSI 

33.213 

3 

11.071 

.899 

.442 

METHOD 

110.202 

3 

36.734 

2.963 

.032 

2-way  Interactions  51.610 

9 

5.734 

.466 

.897 

LSI  METHOD 

51.610 

9 

5.734 

.486 

.897 

Explained 

197.967 

16 

12.373 

1.005 

.452 

Residual 

3262.851 

265 

12.313 

Total 

3460.818 

281 

12.316 

MuRlpie  Range  Test 
Duncan  Procedure 

VarW>te  SCORE 
By  Variable  METHOD 

Variable  SCORE 

G  G  G  G 
r  r  r  r 
P  P  P  P 

Mean  Group         3  4  2  1 

7.3043  Grp3 

7.3803  Qrp  4 

7.5109  Grp2 

8.8116  Qrpi  •  •  • 

C)  Denotes  pairs  ol  groups  significantly  different  at  the  .050  level. 
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Table  9  ANOVA:  General  Satisfaction  bv  Learning  Style  and 
Pn&sentation  Method  with  Spatial  Ability 


•••ANALYSIS  OF  VARIANCE 
FACTOR  SCORE  1  (General  Sattefaclton) 


BY  LSI 

METHOD 
WfTH  VZ2 

Source  (A  Variation 

Sum  (A 
Squares 

DF 

Mean 
Square 

F 

S^nil 

Covarlales 
VZ2 

6.585 
6.585 

1 
1 

6.585 
6.585 

6,941 
6.941 

.009 
.009 

Main  Effedd 
LSI 

METHOD 

2.412 
.812 
1.531 

6 
3 
3 

.402 
.271 
.510 

.424 
.285 
.538 

.863 
.83a 
.657 

2«way  Interactlona 
LSI  METHOD 

20.594 
20.594 

9 
9 

2.288 
2.288 

2.412 
2.412 

.012 
.012 

Explained 
Roslduat 
Toto! 

29.590 
251.410 
281.000 

16 
265 
281 

1.849 
.949 
1.000 

1.949 

.017 

Table  1 1  ANOVA:  Disorientation  effect  by  Learning  Style 
and  Presentation  Method  with  Spatial  Ability 


••■ANALYSIS  OF  VARIANCE""" 

FACTOR  SCORE  3  (Dteortenlalton  ettect) 
BY  LSI 

METHOD 
WITH  VZ2 

Sum  ol         Mean  Slonif 
Source  cH  Variation  Squares  OF  Square       F        of  F 


Tolal 

281.000 

281 

1.000 

Covariates 

6.680 

1 

6.680 

7.007 

.009 

VZ2 

6.680 

1 

6.680 

7.007 

.009 

Main  Effects 

15.476 

6 

2.579 

2.706 

.014 

LSI 

1.671 

3 

.557 

564 

.626 

METHOD 

13.782 

3 

4.594 

4.819 

.003 

2'Way  Interactions 

6.233 

9 

.693 

.727 

.664 

LSI  METHOD 

6.233 

9 

.693 

.727 

.664 

Explained 

28.308 

16 

1.774 

1.861 

.024 

Residual 

252.612 

265 

.953 

Total 

281.000 

281 

1.000 

MuRlpIe  Range  Test 
Duncan  Procedure 

variable  FACTOR  SCORE  3 
By  Variable  METHOD 

Variable  FACTOR  SCORE  3 


Mdan  Group 
-.3444 
-.0163 

.0727 

.2993 


Grpi 
Grp2 
Qrp  3 
Grp4 


G 
r 

P 
1 


(")  Denotes  pairs  of  groups  signltlcantfy  ditterent  at  the  .050  level 


Table  13  Summary  of  p  values 


Learning 
Outcome 

General 
Satis- 
faction 

lext 
Structure 

Disorien- 
tallon 

Material 
Content 

Learning 
Styles 

.442 

.836 

.507 

.626 

.900 

Spatial 
Ability 

.455 

.009* 

.097 

.009* 

.122 

Presenlatlor 
Method 

.032* 

.657 

.001* 

.003- 

.001  • 

Interaction 
Effects 

.897 

.012* 

.574 

.684 

.164 

Table  10  ANOVA:  Satisfaction  with  Text  Structure  by 
Learning  Style  and  Presentation  Method  with  Spatial  Abiiitv 


•••ANALYSIS   OF  VARIANCE 

... 

FACTOR  SCORE  2  (Text  Structure) 
BY  LSI 

METHOD 
WITH  VZ2 

Sum  ol  Mean 
Source  of  Variation  Squares    DF  Square 

F 

SIgnlf 
ofF 

Covariates 
VZ2 

2.642  1 
2.642  1 

2.642 
2.642 

2.775 
2.775 

.097 
.097 

Main  Effects 
LSI 

METHOD 

18.774  6 
2.224  3 
16.805  3 

3.129 

./4i 

5.602 

3.286 
.778 
5.083 

.004 
.507 
.001 

2*way  Interactions 
LSI  METHOD 

7.250  9 
7.250  9 

.806 
.806 

.846 
.846 

.574 
.574 

Explained             28.666  16 
Residual             252.334  265 
Total                  281.000  281 

1.792 
.952 
1.000 

1.882 

.022 

Multiple  Range  Test 
Duncan  Procedure 

Varlabte  FACTOR  SCORE  2 
By  Variable  METHOD 

Variable  FACTOR  SCORE  2 

G  G  G 

G 

Mean  Group 
-.4089      Qrp  1 
.0700  Grp2 
.1315  Grp4 
.2272  Grp3 

1    1  1 
P  P  P 
1  2  4 

r 

P 
3 

(")  Denotes  pairs  of  groups  significantly  different  al  the 

.050  (eve! 

Table  1 2    ANOVA.  Satisfaction  with  Material  Content 
bv  Learning  Style  and  Presentation  Method 
with  Spatial  Ability 


'""ANALYSIS  OF  VARIANCE*"* 

FACTOR  SCORE  4  (Sallsfacllon  wKh  Material  Content) 
BY  LSI 

METHOD 
WITH  VZ2 


Sum  ot 

Mean 

SIgnlf 

Source  oi  Varlatk>n 

Squares 

DF 

Square 

F 

oTf 

Covarlales 

2.270 

1 

2.270 

2.405 

.122 

VZ2 

2.270 

1 

2.270 

2.405 

.122 

Main  Effects 

16.259 

6 

2.710 

2.871 

.010 

LSI 

.552 

3 

.184 

.195 

.900 

METHOD 

15.656 

3 

5.219 

5.530 

.001 

2-W&/  Interactions 

12.376 

9 

1.375 

1.457 

.164 

LSf  METHOD 

12.376 

9 

1.375 

1.457 

.164 

Explained 

30.905 

16 

1.932 

2.047 

.011 

Residual 

250.095 

265 

.944 

Total 

281.000 

281 

1.000 

Multiple  Range  Test 
Duncan  Procedure 

Variable  FACTOR  SCORE  4 
By  Variable  METHOD 

Variable  FACTOR  SCORE  4 


Moan  Group 
-.3327    Qrp  1 
-.0722    Qrp  2 
.0865  Grp3 
.3282    Qrp  4 


Q 
r 
P 
3 


n  Denotes  pairs  of  groups  stgnHlcantly  different  at  the  .050  tevel 
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Strategies  for  Utilizing  Hypermedia:  Instruction  vs.  Information  Access 

Xiaoming  Zku 
Electronic  Data  Systems  Corporation 


Abstract  When  hypermedia  is  demonstrating  its  amazing  capabilities,  its  drawbacks 
such  as  disorientation,  cognitive  overhead,  and  content  deficit  for  instruction  and 
information  access  are  identified.  The  techniques  and  strategies  for  relieving  these 
effects  have  been  recommended  for  the  hypermedia  applications  as  a  v^hole.  This 
article  discusses  the  strategies  based  upon  characterizing  the  differences  betv/ecn  the 
use  of  hypermedia  for  instruction  and  the  use  of  hypermedia  for  information  access, 
v/hich  the  author  believes  to  be  vitally  important  for  the  appropriate  utilization  of 
the  new  technology.  For  information  access,  integrated  devices  of  orientation  is  the 
solution.  For  instruction,  however,  instructional  predominance  that  consists  of  the 
instructional  body  centralized  and  sequential/hierarchical  navigation  schemata  is  the 
key.  The  two  projects  in  which  the  techniques  and  strategies  were  applied  for  each  of 
the  two  hypermedia  application  areas  are  described  as  the  examples  to  support  the 
discussion. 


Introduction 

Disorientation  and  cognitive  overhead  of  the  user  in  hypermedia  system  have  been 
identified  as  the  major  drawbacks  since  the  early  proUferation  of  the  hypertext  and 
hypermedia  applications  (Conklin,  1987). 

Hypertext  or  hypermedia  starts  with  the  assumption  that  text  or  any  other  piece 
information  can  be  separated  into  items,  and  that  reading  means  tracing  the 
relations  between  concepts  which  link  the  items  (Kommers,  1990).  Hyperspace 
browsers  can  easily  get  lost  within  the  structure,  forgetting  where  they  are  and  how 
they  arrived  there.  They  need  additional  mental  "effort"  to  maintain  simultaneous 
traversals  through  a  hypertext  system  (Furata  &  Stott,  1988, 1989). 

Rather  than  the  disorientation  and  cognitive  overhead,  content  cohesion  deficit  is  a 
more  conspicuous  problem  when  applying  hypermedi.\  for  instruction. 

For  instruction,  the  designer  has  attempt  to  structure  the  information  presentation 
in  a  way  that  he  or  she  believes  to  be  the  best  for  the  learner  to  learn.  The  context 
weighs  no  less  important  than  the  information  itself.  The  nature  of  hypermedia, 
however,  that  allows  the  information  items  being  separated  and  the  related  or 
unrelated  items  being  branched  makes  for  a  potentially  choppy  exposure  to 
information,  with  the  consequence  that  cohesion  between  the  elements  making  up 
the  whole  of  what  the  learner  is  exposed  to  may  be  deficient.  Comprehension  will 
potentially  suffer  if  that  is  the  case  (Duchastel,  1990,  Marchionini,  1988). 
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In  this  article  I  will  discuss  techniques  and  strategies  for  solving  these  problems 
based  upon  characterizing  the  differences  between  the  use  of  hypermedia  for 
information  access  and  the  use  of  hypermedia  for  instruction.  Clarifying  the 
differences  is  vitally  important  for  the  appropriate  utilization  of  the  new 
technology.  Two  projects  in  which  the  techniques  and  strategies  were  applied  in 
each  of  the  two  hypermedia  application  areas  are  presented  later  in  the  article  to 
support  the  discussion. 

Utilizing  Hypeimedia  for  Information  Access 

Information  access  has  been  the  biggest  area  in  which  hypermedia  technology  is 
applied.  Retrieving  information  from  a  database,  searching  facts  in  a  information 
system,  using  an  computer  application  package,  and  even  using  a  computer 
operating  system  all  possibly  features  hypermedia  application.  The  characteristics 
that  the  readability  of  information  piece  is  more  important  than  the  validity  of  its 
outgoing  relations  of  information  access  and  the  decontextualized  nodes  and  links 
nature  of  hypermedia  make  the  two  a  perfect  match.  This  is  one  of  the  reasons  that 
hypermedia  has  gained  the  great  attention  and  development  in  the  field  of 
information  access  since  it  was  originated. 

With  the  technology  breaking  through,  hypermedia  is  bringing  forth  a 
unprecedented  navigational  topologies  in  which  the  linear,  hierarchy,  hypercube  or 
hypertorus,  dag,  and  arbitrary  browsing  are  possible  (  Parunak,  1989).  Hypermedia 
becomes  a  fundamental  technique  for  both  the  authors  and  users  to  build  and  to  use 
the  system  for  information  exchange  and  communication  purposes. 

The  user's  hyperspace  disorientation  and  cognitive  overhead  is  a  side  effect  of  the 
powerful  navigation.  It  is,  however,  not  a  fundamental  problem  of  using 
hypermedia  for  information  access.  Imagining  when  tourists  get  lost  in  a  big  city 
where  they  have  never  been  to  before,  they  could  hardly  blame  either  the  urban 
planning  or  the  transportation  system  of  the  city  although  the  improvement  of  the 
city  orientation  such  as  providing  more  traffic  sign,  city  map,  or  tour  guide  is  always 
preferred.  The  same  hold  true  for  using  hypermedia  for  information  access,  clear 
orientation  devices  for  guiding  the  navigation  are  always  beneficial  to  the  users 
(Nielsen,  1989;  1990).  This  should  introduce  the  following  strategy  for  the 
improvement. 

The  strategies  is  to  create  a  transparent  user  interface.  Through  this  interface,  the 
users  know  where  they  are,  where  they  have  been,  and  where  they  can  go  next.  For 
example,  an  overview  diagram  can  provide  some  of  the  same  assistance  to  a 
hyperspace  browser  as  a  map  provided  to  a  tourist.  The  following  are  a  list  of  the 
techniques  and  actions  for  creating  the  transparent  interface. 

Tour  Map:  keep  a  map  in  the  system  that  can  be  accessed  any  time  from  anywhere. 
The  map  shows  all  the  significant  nodes  and  the  links  of  the  system,  very  similar  to 
a  city  map  that  shows  all  the  buildings  and  the  traffic  ways. 
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Footprint  :  present  a  label  in  the  map  to  indicate  where  the  browser  is  in  the  system. 
Better  than  that  a  tourist  who  may  need  to  ask  someone  in  order  to  know  where  he 
or  she  is,  hyperspace  browser  can  get  the  information  quickly  and  automatically. 

Zoom  and  Roam  Facility  :  Add  zoom  and/or  roam  functions  in  the  tour  map  or 
other  local  maps.  By  using  zoom,  the  browser  can  see  more  or  less  detail  the  map  or 
part  of  the  map  through  clicking  the  zoom  in  or  out  button.  By  using  roam,  the 
browser  can  let  any  part  of  the  map  displayed  on  the  full  screen  (Beard  &  Walker 
1990), 

Compass  :  Include  the  label  or  function  in  the  system  that  always  directs  the 
browser  to  the  nodes  that  are  important  to  them  (Bernstein,  1988).  This  label  should 
be  dynamically  updated  according  to  the  node  the  browser  is  on  currently. 

Fisheye  View:  Enhance  the  tour  map  so  that  the  entire  information  space  is  shown 
on  a  single  overview  diagram  but  in  varying  level  of  detail  by  using  the  hierarchical 
chart  (Furnas,  1986).  This  function  can  be  integrated  into  the  zoom,  or  vice  versa,  so 
that  the  overview  diagram  become  much  powerful. 

Landmark:   Create  dynamically  the  labels  that  define  an  usable  structure  as  part  of 
the  browsing  process  by  highlighting  the  milestone  nodes  in  the  tour  map. 

Notebook:    Make  it  possible  to  allow  the  browser  to  record  the  milestones  of  their 
browsing  process  (Slaney  1988).  This  function  is  similar  to  landmark  but  different  in 
that  the  notebook  allows  browser  to  record  the  milestone  for  multiple  paths  and 
allows  the  browser  to  compare  one  to  another. 

Besides  the  transparent  interface  strategy,  author  defined  navigation  paths  are  also 
helpful  for  users  to  keep  away  from  disoriented  or  cognitive  overloaded. 
Researches  call  it  paths  strategy  (Trigg,  1983;  Canter,  1985;  Hammond  &  AUinson, 
1988;  Marshall,  1989;  Zellweger,  1989).  Paths  strategy  gives  the  browser  a  procedural 
description  of  how  to  get  to  the  target.  Similar  to  the  direction  given  to  a  tourist, 
"turn  left  at  the  first  light  and  right  at  the  second  gas  station,  and  it's  the  blue  house 
on  the  right"  for  example,  a  direction  for  finding  the  desired  node  can  be  given  to  a 
browser  by  the  paths.  A  path  can  be  very  simple,  linear  path,  sequential  path  for 
example.  A  path  can  also  be  very  complicated  such  as  conditional  or  arbitrary  path. 
Path  gives  author  the  control  to  some  degree  on  how  the  information  should  be 
accessed.  Author  should  leverage  the  control  according  to  the  complexity  of  the 
system  and  the  familiarity  of  the  user  to  the  system.  A  completed  path  consists  of 
the  links  mechanism  and  the  orientation  information  to  the  browser.  The  way  to 
author  the  path  depends  on  the  hypermedia  system  used.  When  using  HyperCard 
as  the  authoring  environment,  for  example,  Hypertalk  script  is  the  basic 
mechanism  for  creating  the  path  and  the  message  on  the  cards  will  be  the 
procedural  description  of  the  navigation. 


Utilizing  Hypermedia  for  Instruction 
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Using  hypermedia  for  instruction  is  a  natural  progress  of  computer-mediated 
instruction  for  its  promise  of  ease  of  use,  richness  in  presentation,  and  flexibility  of 
hypermedia.  The  similar  progress  was  made  about  twenty  years  ago  when 
programmed  instruction  was  shifting  to  computer-mediated  instruction  (Anglin, 
92).  The  technology-mediated  instruction  history  shows  thr;  the  instruction  and 
technology  share  joys  and  sorrow,  weal  and  woe. 

The  educational  potential  of  hypermedia  is  in  connecting  concepts  by  following 
paths,  thus  forming  interpretation  and  synthesizing  information  (Marchionini, 
1988).  This  potential,  however,  finds  no  ground  in  traditional  instructional  design 
paradigm  in  which  instruction  is  defined  as  a  series  of  prescribed  events  and 
activities  for  achieving  specified  learning  objectives.  Hypermedia  provides  the 
learner  with  the  opportunities  to  diverge  from  the  prescribed  instructional  path  and 
sequence;  to  juxtapose  text,  animation,  and  sound;  to  turn  the  technology  back  on 
itself  as  an  aid  in  reviewing,  studying,  and  producing  new  interpretations  of  the 
content  (Marchionini,  1988).  As  a  result  of  this,  cohesively  organized  contents 
become  deficient,  freedom  becomes  chaos.  That's  why  hypermedia  and  formal 
learning  are  considered  to  be  conceptually  distant  from  one  another;  hypermedia  is 
more  appropriate  for  situated,  associational,  or  discovery  learning  in  which  no 
learning  outcomes  are  previously  specified  (Duchastel,  1990). 

The  orientation  devices  approach  for  releasing  the  user's  "lost  in  hyperspace" 
recommended  in  last  section  of  this  article  is  not  a  fundamental  solution  of  the 
problem.  An  new  theory  for  interpreting  the  cognitive  process  and  revealing  the 
law  of  learning  with  hypermedia  may  be  a  solution.  The  theory  should  be 
sophisticated  enough  to  prescribe  the  best  course  of  actions  for  using  and  evaluating 
hypermedia  for  human  learning.  In  this  solution,  content  cohesion  deficit  is 
approached  in  a  totally  different  way  in  which  discovery  learning,  situated  learning, 
or  learning  by  making  mistakes  is  considered  to  be  the  dominant  transactions  for 
human  learning.  In  order  to  find  the  solution,  knowledge  representation  of 
hypermedia  that  addresses  both  issues  of  readability  of  the  text  item  and  the  validity 
of  outgoing  relationship  need  to  be  investigated.  Explicating  the  mental  effects  of 
knowledge  elicitation  and  knowledge  acquisition  by  hypertext  to  reveal  the  pattern 
of  learning  with  hypermedia  is  necessary.  Hypermedia  learning  is  creating  a 
exciting  research  field,  no  doubt,  but  its  current  premature  study  methods,  very 
limited  research  findings,  and  unproven  cognitive  validity  can  hardly  provide 
convincing  and  practical  solution  for  the  design  of  an  effective  instruction  (Mayers, 
1990). 

A  practical  and  effective  approach  is  to  reduce  the  requirement  for  arbitrary 
navigation  by  setting  a  instructional  body  centralized  predominant  path  to  guide  the 
learner  in  navigation  (Zhu,  1992).  This  approach  consists  of  two  basic  instructional 
navigation  schemata.  First,  it  allows  learner  to  navigate,  primarily,  within  the 
instructional  dominance  as  well  as  between  the  dominance  and  their  auxiliary 
components  such  as  the  course  map,  course  glossary,  references,  etc  (Figure  1). 
Second,  within  the  instructional  dominance  ,  it  combines  sequential  and 
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hierarchical  navigation  paths  that  differentiate  information  from  one  level  to 
another  (Figure  2). 


6    ooo    o  oo 

^    Instojctional dominance 
^ '       Instructional  auxiliary 

Figure  1.  Instojctional  body  centralized  navigation  schema 


Figure  2.  Instructional  dominance  consists  of  sequential  and  hierarchiocal 
navigation  paths 


There  are  two  major  characteristics  of  this  approach.  First,  it  conforms  to  the 
research  finding  that  learners  in  learner  control  treatment  have  regularly  learned 
less  than  learners  those  in  treatment  controlled  by  the  instructor  or  an  adaptive 
instructional  design  (Jonassen,  1986;  Hannafin,  1989).  Second,  it  effectively  utilizes 
hypermedia  technology  through  which  learner  is  able  to  obtain  the  information  that 
aids  his  or  her  learning.  Balancing  between  system  control  and  learner  control  to 
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maximize  the  benefits  and  reduce  the  risk  of  using  hypermedia  for  instruction  is  the 
purpose  as  well  as  the  end  result  of  this  approach. 


Project  Examples 

In  this  section,  I  will  present  two  projects  in  which  the  strategies  discussed  above 
were  applied  in  the  two  different  hypermedia  application  areas.  They  should  serve 
as  the  examples  to  support  or  elaborate  the  discussion.  The  table  below  presents 
general  information  about  the  two  projects.  More  description  is  provided  as 
follows. 

In  the  first  project,  information  access  is  the  type  of  hypermedia  application.  The 
system  provides  the  user  with  the  information  about  the  library  materials,  library 
services,  computerized  systems,  etc.  The  information  is  independent  one  from 
another.  Users  don't  need  any  specific  prescribed  path  or  sequence  because  they 
access  the  information  based  on  their  own  interests.  Paths  and  transparent  interface 
strategies  were  applied  in  authoring  the  system.  Linear,  hierarchical,  and 
backtracking  path  strategies  were  used  to  organize  the  information. 


Project 
Name 

Type  of  Application 

Strategies  Applied 

Authoring 
System  Used 

Library 

Information 

Systenn 

Information  system  that  allows 
the  user  to  access  the 
information  about  the 
university  library  such  as 
circulation,  reference,  document, 
map,  computerized  catalog 
(LUIS),  indexes,  subject 
headings,  services,  materials 
and  locations,  etc. 

Transparent  Interface 

•  Tour  Map 

•  Footprints 

•  Zoom  Facility 

•  Fisheyes  View 

•  Landmark 

Paths 

•  Linear  path 

•  Hierarchical  path 

•  Backtracking  path 

•  Conditional  path 

HyperCard 
2.1 

SLC-An 
Introduction 

An  instructional  system  that 
teaches  the  basic  facts  and 
concepts  of  the  Systems  Life 
Cycle  (SLC)  methodology  that 
is  used  for  developing  and 
maintaining  information 
technology  systems. 

Instructional  lesson  centralized 
navigation  schema  that  provides 
the  learner  with  a  guided  tour  on 
the  prescribed  instructional  path 
and  sequence.  At  the  same  time, 
the  learner  is  allowed  to  access 
other  components  relevant  to  the 
instruction  such  as  glossary,  course 
map,  course  orientation  using 
hypermedia  as  long  as  he  or  she 
stays  in  the  predominant  path. 

Authorware 
Professional 
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and  at  the  same  time,  to  facilitate  the  navigation.  A  map  that  has  the  features  of 
footprints,  zoom,  fisheyes  view,  and  landmark  was  built  into  the  system.  It 
provides  users  with  an  instant  help  regarding  v/here  they  are,  where  they  have  been 
to  and  where  they  can  go  in  the  system. 

In  the  second  project,  instruction  is  the  key.  The  system  was  designed  to  teach  the 
basic  facts  and  concepts  of  the  methodology.  The  learner,  instead  of  the  user,  is 
expected  to  achieve  specified  learning  objectives  upon  completion  of  the 
instruction.  Based  on  the  task  analysis,  it  was  also  believed  that  certain  learning 
path  and  sequence  are  important  to  the  learning.  In  order  to  meet  these 
requirements,  two  major  navigation  schemata  were  applied  using  hypermedia 
technology.  One  was  the  instructional  lesson  centralized  navigation.  The  other  was 
the  combination  of  linear  and  hierarchical  presentation.  The  first  creates  a 
predominant  path  for  the  instruction,  avoiding  its  content  cohesion  deficit.  The 
second  makes  it  possible  that  information  can  be  presented  in  either  a  sequence  or  a 
optional  manner,  so  that  the  presentation  can  be  more  effective  and  efficient. 

Conclusion 

The  biggest  strategy  for  effectively  utilization  of  hypermedia  is  to  differentiate 
between  instruction  and  information  access  and  differentiate  between  task-oriented 
learning  and  situated  learning.  There  is  no  fundamental  problem  with  using 
hypermedia  for  information  access.  User's  hyperspace  disorientation  and  cognitive 
overhead  can  be  overcome  through  the  integrated  devices  of  orientation.  However, 
there  is  a  fundamental  problem  with  using  hypermedia  for  instruction.  It  is  the 
instructional  content  cohesion  deficit.  Although  an  unknown  theory  may  help 
interpret  hypermedia  learning  better  and  solve  the  problem,  the  practical  and 
effective  solution  to  the  problem  is  to  lead  the  learner  to  a  predominant  navigation 
path  in  which  the  instructional  body  of  the  system  is  centralized. 
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Considering  Organizational  Culture  In  Selecting  CBT  Development  Software 

Robert  C.  Fratini,  AT&T  National  Product  Training  Center 


There  are  a  handful  of  software  best-sellers  for  applications  such  as  word  processing  or 
spreadsheets.  However,  a  recent  issue  of  CBT  Directions  lists  will  over  a  hundred  authoring 
languages  and  systems  for  use  in  the  development  of  computer-based  training  (CBT),  Why  so 
many,  with  new  ones  coming  on  to  the  market  every  year? 

The  hypothesis  explored  in  this  presentation  is  that  the  level  of  satisfaction  with  a  piece  of 
software  that  is  experienced  by  an  organization  (specifically,  an  organization  involved  in  the 
development  of  CBT)  depends  on  how  closely  the  functionality  expressed  in  that  software 
matches  with  that  organization's  values. 

During  the  presentation,  based  on  discussion  and  input  from  the  attendees, k  I  will  discuss  how 
major  areas  of  software  functionality  expressed  in  CBT  development  tools  can  be  assessed 
against  such  values  of  an  organization's  culture  as: 

-Individual  need  for  control 

-Level  of  trust  of  others 

-Attitudes  towards  the  systematic  instructional  development  process 

-Attitudes  towards  organizational  standards 

-Attitudes  towards  the  place  of  "entertainment"  during  instruction 

-Reviewers'  feelings  of  autonomy  prior  to  the  general  release  of  instructional  maftsiials 

-Attitudes  towards  student  or  user  control  during  training  delivery 

Attendees  will  be  able  to  determine  the  level  of  consistency  between  functionality  expressed  in 
current  and  potential  CBT  development  software  and  the  culture  and  values  of  their  developers. 

Lifelines  for  the  Struggling  Multimedia  Developer:  Living  on  Multimedia's  Bleeding  Edge 

Jeanne  Gleason,  New  Mexico  State  University 

Even  during  financial  crisis  and  piogrammatic  cutbacks,  effective  multimedia  presentations  can 
be  developed. 

This  presentation  will  cover  how  to: 

-cope  with  limited  funding  for  equipment  and  personnel, 

—deal  with  unrealistic  administrative  expectations, 

-develop  clear,  attainable  goals, 

-know  when  to  play  it  safe  and  when  to  take  a  risk, 

-build  strong  design  and  development  techniques, 

-increase  productivity  by  building  "ownership", 

-turn  deadline  pressure  into  creative  energy,  and 

-effectively  implement  and  marker  the  outcome. 
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Examples  will  be  drawn  from  The  Natural  Resource  Extravaganza,  a  multimedia  presentation 
for  public  environments  developed  in  90  days  with  extremely  limited  funds  and  student 
assistance.  The  speaker  developed  a  model,  called  ADOBE,  to  manage  activities.  She  will 
share  her  development  tricks  to  keep  the  project  on  track,  and  how  using  development  templates 
can  build  administrative  support  while  £uiding  parallel  development. 

This  system  was  developed  for  the  PC  environm^^nt,  using  touch  screen  input,  a  laser  disc, 
computer  graphics,  and  the  IMSATT  programming  language.  The  presenter  is  currently 
developing  additional  systems  using  Macintosh  systems  for  The  Smokey  Bear  Center,  the 
USD  A,  and  the  Navajo  Indian  Tribe, 
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Quality  Improvement  Cycle  (QIC)  and  Computer  Supported  Learning  in  administrative 
organizations.  An  observation  through  some  experiments. 


Gianpicro  Bussolin 

University  of  Torino  ( Italy) 

The  discussion  concerns  the  nature  and  above  all  ihe  experimental  issues  resulting  from  models, 
systems  and  projects  in  progress  by  the  Author  for  the  Computer  Supported  Learning  .  The 
methodology  of  Quality  Improvement  Cycle  (see  Total  Quality  Management)  is  considered  in  order 
to  improve  and  enhance  tlie  existing  processes  of  learning  about  problems  of  an  administrative 
nature  (see  also  management  education).  H.A.Simon*s  ideas  are  a  starting  point  for  many 
observations  .  Further  references  shall  be  given  in  the  paper  and  during  the  oral  presentation  (*X 


Preface 

This  paper  contains  some  key  words  which  require 
a  particular  explanation. 

The  word  "experiment"  will  often  be  used  to 
indicate  that  the  projects  and  systems  discussed  are  to  be 
considered  in  their  experimenial  nature,  in  order  to  draw 
from  ihem  a  "corpus"  of  empirical  observations  and 
comments  upon  their  factual  rcsulLs.  These  results  and 
these  observations  may  serve  to  reinforce  more  general 
theories  and  scientific  principles,  or  may  be  of  a 
controversial  nature  as  regiuds  these  very  principles  and 
theories  mentioned  above. 

The  word  "system"  will  often  be  employed  in  order 
to  indicate  a  "  set  or  lu^rangement  of  things  so  related  or 
connected  as  to  form  a  unity  or  organic  whole  "  (see 
Webster's  New  World  Dictionary  of  the  American 
Language),  and  ,we  say  above  all,  having  a  "purpose" 
which  plays  the  important  role  of  molding  this 
arrangement  during  the  process  of  its  construction. 
(H.A.  Simon,  1978  ,  "Preface"). 

This  term  is  often  used  in  our  everyday  language 
even  though  it  has  a  rather  vague  nature  with  a  rather 
weak  meaning.  In  mis  paper  this  term  will  be  used  to 
indicate  that  this  arrangement  of  things  is  considered 
both  in  its  prqjectual  condition  and  in  its  state  of 
supporting  practical  applications  and  results.  This  ar- 
rangement of  things  will  obviously  include  not  only 
projectual  activities  but  also  hard  and  software ,  such  as 


(*)  Experiments  b)  c)d)  and  the  tlicoretical  research  into 
the  field  of  the  Sciences  of  the  Anificial  has  found  support 
from  the  Italian  "Minislcro  dclla  Universitii  e  dcHa  Ricerca 
Scientifica"  and  "Consiglio  Na/.ionalc  dellc  Ricerche". 


information  technology  may  offer  with  its  computers, 
languages  and  programs.  The  word  "  model"  will  be 
used  in  its  definite  meaning  of  representation  and 
design. 

The  paper  is  divided  into  two  parts.  Part  I  contains 
a  short  explanation  of  some  systems  concerning  ,  in 
some  ways,"  computers  and  learning  " ;  Part  II  contains 
some  observations  upon  these  systems,  in  their 
experimental  aspect  as  previously  made  clear. 


Part  I 

a)  The  first  experiment,  from  an  historical  point 
of  view,  has  as  scenario  or  outline  of  ihe  organizational 
and  administrative  context,  a  project  for  a  computer- 
based  system  in  a  worldwide  company  .  This  project 
concerns  tlie  planning,  manufacturing  and  distribution 
of  its  outputs. 

We  are  in  the  60's  and  the  computer-based 
Management  Information  System  has  the  main  purpose 
of  integrating  the  administrative  functions  above 
mentioned  ,  with  a  particular  regard  for  the  distribution 
function  of  products  over  the  home  market  (peripheral 
warehouses)  and  the  foreign  markets.  We  might  also 
say,  with  the  use  of  an  up-to-date  concept,  that  the  goal 
is  "  total  quality"  in  terms  of  its  attributes,  namely  that 
concerning  customer  service.  The  essence  of  this  short 
description,  is  necessary  for  two  main  reasons:  the 
highly  complex  nature  of  this  project,  and  the 
organ i/.ational  cliangc  entailed  in  the  project  itself. 

The  generation  of  new  information  technologies 
(new  products  of  hard  and  software)  has  always  played 
a  role  of  change  within  administrative  organizations. 
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imposing  a  new  form  of  informiUion  processing  .  The 
success  and  often  the  failure  of  these  computer-based 
MIS's  has  strong  links  with  the  administrative 
knowledge  of  those  individuals  involved  in  the  project 
itself. 

We  use  the  expression  "administrative  knowledge  " 
to  recall  a  complex  problem  where  it  is  difficult  to 
disentangle  concepts  like:  communication,  learning, 
education,  skill  and  expertise,  behavior,  etc. 
(H.A.Simon, 1961, Chapter  VIll  "Communication")  . 
This  project  of  computer-based  MIS  has  been  studied 
in  the  light  of  this  administrative  knowledge  and 
therefore  in  the  light  of  administrative  education, 
arriving  at  the  construction  of  a  model  of  computer- 
based  simulation  for  the  administrative  individual  in- 
volved in  the  MIS  project  itself.  This  MIS  project  was 
so  complex  that  an  attempt  was  made  to  draw  from  the 
design  of  the  project  that  form  of  "integrated 
knowledge"  which  tl\e  administrative  individual  had  to 
possess  in  order  to  master  the  MIS  successfully 
(Bussolin,  1974) . 

The  skill  and  expertise  of  this  individual  are 
supposed  to  be  enriched  through  computer-interactive 
learning  upon  hypothetical  situations  generated  by  the 
simulation.  This  simulation  model  (Bussolin,1967) 
may  be  considered  a  base  for  a  computer-  supported 
administrative  or  management  education  project 
contained  within  the  broader  MIS  project .  At  the  core 
of  the  theoretical  design  of  this  model  there  is  a  so- 
called  "logical  unit  of  discourse  "  for  die  representation 
of  the  product  which  is  increasingly  and  dynamically 
enriched  with  logical  attributes  on  the  assembly  lines  of 
manufacturing  and  on  the  lines  of  distribution  by  truck 
transportation  to  the  peripheral  warehouses.  This 
"logical  unit"  may  be  considered  as  the  smallest  brick  ol 
the  design  in  order  to  represent  dynamically  what  may 
happen  to  the  output  through  the  operation  of  the 
proposed  MIS  . 

This  model  reproduces  a  shape  of  parallelism  of 
administrative  processes  superimposed  on  physical 
processes  (assembly  lines  of  manufacturing  and 
iransporunion  lines).  The  source  code  for  the  computer 
programs  was  COBOL  .  We  should  not  forget  that  we 
are  in  the  60's  .  Today  ,  with  the  conceptual  tools 
offered  by  Artificial  Intelligence  and  object-oriented 
languages,  we  might  design  and  rcali/.e  in  a  more 
sophisticated  manner  these  very  models  for  education. 
This  kind  of  models  do  not  exhaust  their  function  in  a 
pure  represeniation  of  physical  or  administrative  facts, 
but  they  must  possess  an  interface  of  communication 
with  the  individual  who  must  learn  through  a 
computer- interactive  human  activity.  (Even  if  dealing 
with  a  different  kind  of  knowledge  base,  for  this  basic 
idea  of  "interface"  for  ihc  "learner"  sec  Poison- 


Richardson  ,(1988)  ;  and  one  of  the  first  and 
fundamental  theoretical  discussions  in 
Simon,(l978)Chapter  "Understanding  the  natural  and 
the  artificial  world  "). 

Within  this  model  we  have  a  shape  of  "dynamic 
represeniation"  which  allows  the  learner  to  make  his 
decisions  at  a  given  point  in  time  and  to  observe  the 
state  of  affairs  of  his  system  after  a  supposed  interval  of 
time.  This  process  is  obviously  repeated  ,  is  recursive, 
along  the  course  of  simulation. 

The  nature  of  learning  of  this  administrative  ed- 
ucation which  may  be  allowed  by  these  tools  is 
basically  different  if  compared  with  traditional  tools. 
The  "integrated"  administrative  knowledge  descends 
from  the  fact  that  the  learner  is  allowed  to  master  a 
wider  span  of  administrative  situations  ,  and  if  the 
experiment  is  carried  on  in  a  cooperative  team,  this  form 
of  learning  may  reach  deep  within  the  individual.  The 
technological  support  of  computer  networks  has  (see 
further  experiments)  wide  potentialities  in  this  last 
sense. 

b)The  project  of  point  a)  has  not  been  implemented 
in  a  real  context  of  learning,  and  our  research  regards 
simply  the  design  of  its  model.  On  the  contrary,  for  this 
next  project  (Bussolin,  1979),  we  may  report  on  some 
results  deriving  from  iLs  application  in  a  real  human 
context  of  university-  level  classes  for  industrial 
administration  teaching. 

This  new  model  employs  Forrester's  method- 
ology of  level  and  rates  equations"  and  the  source  code 
of  the  computer  program  is  FORTRAN  .  It  does  not  use 
the  compiler  DYNAMO  .  In  fact ,  the  model  under 
discussion  does  not  reflect  the  philosophy  set  forth  by 
various  applications  of  Fonester's  methodology  .  The 
use  of  this  methodology  consists  in  the  representation 
of  given  phenomena  (for  instance  concerning 
economics)  and  the  related  study  of  the  system  's 
behavior  for  a  given  interval  of  time.  On  the  contrary, 
our  model  has  an  educational  purpose  ,  i.e.  the  teaching 
of  basic  concepts  of  industrial  adminisU'ation,  and  this 
purpose  corrcctly  molds  the  whole  design. 

In  this  project,  each  group  of  students  makes 
decisions  about  its  company  and  may  observe  their 
results.  Of  course  the  cognitive  process  is  recursive  in 
its  feed-back,  in  the  sense  that  we  have:  decisions- 
rcsults,decisions-results,  etc.  The  same  mentioned 
feature  of  learning  in  an  evolutionary  context  by 
interactive  computer  simulation  mentioned  in  the  model 
of  point  a)  is  contained  in  the  model  under  discussion. 

The  administrative  system  is  subdivided  into 
subsystems  like  :  finance,  production,market,etc.  linked 
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by  channels  in  which  flow  various  kinds  of  units  : 
money,  invcsiinenis,  products,  etc.  A  kind  of 
simultaneity  occurs  in  the  dynamic  flowing  of  units  into 
the  various  channels,  similar  to  what  we  have  discussed 
in  point  a).  The  learning  process  of  the  student  is 
therefore  like  that  illustrated  at  point  a)  where  we  have 
dealt  wilh  an  "integrated  knowledge".  We  might  also 
say  lhat  the  learner  acquires  a  "process  knowledge",  i.e. 
a  knowledge  of  the  "how's"  rather  than  a  knowledge  of 
the  "  what's"  .  The  first  iorm  of  knowledge  concerns 
"how"  administration  processes  are  carried  on  ,  and  the 
second  deals  wiili  "what"  these  processes  are  (Bussolin, 
1974.1986). 

It  has  been  possible  to  compare  these  two  Conns  of 
knowledge  in  a  human  context  of  teaching 
administration  matters  to  students  at  university  levels, 
the  first  one  being  much  more  highly  motivating  for 
further  investigations  on  the  part  of  these  students.  The 
"role  playing"  of  the  individual  belonging  to  a  group  of 
student :  and  "the  role  playing"  of  the  group  as  a  unit  is  a 
slroui  iason  for  enhancing  all  the  attributes  which 
conce.  learning. 

It  is  important  that  the  designer  of  the  model  be  also 
the  teacher  who  uses  this  very  model  for  educational 
purposes.  In  fact  ,  if  the  design  is  module-based  it 
should  be  ciisy  to  redefine  the  module  of  user  interface 
according  to  the  background  of  the  student  and  the 
eduCiUional  purposes.  We  all  know  how  expensive  it  is 
to  modify  a  computer  package  ,  and  often  it  is 
convenient  to  redesign  the  whole  model  if  it  is  not 
module  based . 

Competition  among  students  is  very  imporumt  in 
the  cognitive  process  of  student  learning.  It  has  been 
possible  to  note  that  the  more  intense  this  competition, 
the  loss  will  be  the  learning  in  depth  ,  particularly  wilh 
regard  to  the  learning  of  interrelations  among  variables 
of  the  system  being  simulated.  We  might  also  say  iJial 
different  human  values  may  be  transierrcd  according  to 
the  greater  or  lesser  intensity  of  this  competition  .  The 
model  under  discussion  does  not  contain  a  high  degree 
of  competition  in  that  the  behavior  of  each  company 
does  not  influence  the  behavior  of  the  other  partners  of 
the  market.  Nevertheless  a  certain  degree  of 
competition  exists  among  groups  ,  in  that  the  set  of 
companies  at  ilie  beginning  of  the  educational  process 
has  the  same  resources  and  the  same  mechanisms 
governing  their  responses  to  decision-making. 

c)The  model  under  discussion  (Rosso- 
Vaglienti,  1989)  concerns  cooperative  learning  for  the 
operation  of  a  bank  systeiii.  This  model  has  the  purpose 
of  stimulating  decision-making  activities  of  the 
individuals  in  a  real  context  of  cooperative  behavior 
with  the  technological  support  of  a  local  area  network  of 


PCs  dedicated  to  the  various  Icams.  Each  group  of  de- 
cision makers  has  the  task  of  operating  one  of  the 
branches  of  the  bank  and  the  coordinator  of  the  learning 
experiment  is  also  the  headquarters  general  manager  of 
this  bank-branches  network  .  This  coordinator.through 
his  PC,  determines  and  supplies  to  the  PCs  of  the 
branch  offices,  the  flows  of  transactions  (interchange  of 
flows  of  debit-credit  items)  which  create  an  informa- 
tion input  for  the  decision  making  of  each  branch 
within  its  local  market.  Each  team  makes  decisions 
based  on  its  flows  of  items  (views  on  the  current 
accounts  files  of  its  own  branch)  while  the  central 
coordinator  has  the  information  support  of  the  global 
dauibase  of  transactions.  The  decision-making  process 
of  the  set  of  teams  is  not  simply  carried  on  through  a 
transmission  of  data  via  the  network  communication 
channels  (such  as  the  electronic  mail  process)  but  also 
and  above  all  through  a  complex  human  process  of 
coordination  in  which  arc  combined 
competition,conflicl  and  sharing  of  purposes  (Malone 
and  Crowslon ,  1990). 

d)  The  last  projects  to  be  dealt  wilh  are  research 
projects  in  progress  and  they  concern  Intelligent 
Tutoring  Systems  (ITS)  for  the  teaching  of  Accounting 
and  Hyper  Text  applications  for  the  leaching  of 
Industrial  Administration  problems . 

The  approach  for  the  first  research  docs  not  imply 
the  use  of  a  hard  support  like  a  text  book  of  Accounting 
and  the  PC  is  not  used  to  verify  the  level  of  skill  of  the 
learner  .  On  the  contrary  ,  this  research  aims  at 
developing  in  the  learner  a  manipulative  ability  in  book- 
keeping and  balance-sheet  generating.  The  word 
"manipulative"  is  here  employed  in  an  abstract  sense 
and  in  a  positive  sense  ,  i  .c.  in  the  sense  of  operating 
and  controlling  skillfully  a  set  of  accounting 
transactions  with  the  purpose  of  arriving  at  a  correct 
input  into  the  accounts  and  a  correct  construction  of  the 
balance  reports.  The  standard  and  normal  applications 
of  CAl  arc  based  on  external  and  hard  copies  of  text- 
books of  Accounting  and  their  purpose  is  to  verify  the 
levels  of  "what"  has  been  learnt,  through  a  variety  of 
instructional  frameworks  (see,  for  example,  the 
methodology  of  the  so  called  "drill  and  practice") .  On 
the  contrary,  the  central  idea  in  our  research  is  that  the 
learner  has  a  series  of  more  or  less  complex  cases  from 
the  PC  ,  and  through  a  tutorial  support  given  by  his  sys- 
tem ,  the  learner  can  manipulate  his  transactions  in  order 
to  accomplish  the  purposes  suggested  by  his  PC. 

Among  the  vcuieiy  of  problems  to  solve,  we  have  to 
lace  the  question  of  "how"  the  human  expert  may  create 
the  knowledge  base  and  to  "what"  extent  reasoning 
ability  has  to  be  assigned  to  the  computer  expert. 
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Research  into  Aniliciiil  Inielligcnce  seems  to  be 
improving  and  to  lay  even  more  stress  on  the  computer 
expert ,  while  the  computer-based  educational  approacli 
seems  to  reaffirm  (this  is  our  philosophy)  the 
imporuince  of  the  human  teacher  in  his  educational 
process.  We  might  remember  at  this  point  of  the 
discussion  the  idea  of  "bounded  raiionaliiy"  (  Simon  , 
1961,  Chapter  IV  "Rationality  in  Administrative 
Behavior").  The  correct  answer  lies....  in  the 
"contingency"  situation,  in  Qie  approach  of  the  "hie  et 
nunc". 

As  regards  the  research  projects  carried  out  on  the 
hypertext  software  for  creating  educational  support  in 
the  teaching  of  industrial  administration  matters  at 
university  levels,  we  are  at  the  very  beginning.  We 
have  encountered  the  obstacles  of  "which"  textbooks 
are  to  be  chosen,  "how"  to  use  this  technology  for 
different  courses  in  cliiTorcni  universities  etc.  The 
choice  of  subjects  to  be  taught  and  Iciirnl  is  an  extremely 
important  choice,  especially  when  the  scientific 
community  has  various  approaches,  various  and 
controversial  point  of  views  ,etc.  These  subjects  are 
quite  different  from  subjects  like  Geography....  As  we 
will  note  in  Part  11,  the  educational  computer-  based 
approach  is  "organization  driven",  we  also  have  to 
consider  the  messages  coming  from  the  environment 
and  we  have  to  define  the  strategies  to  develop  these 
projects. 


Pan  II 

As  previously  said  in  the  "Preface" ,  in  this  "Part  11" 
some  observations  and  comments  will  be  given 
regarding  the  experiments  of  "Part  1"  and  their  related 
scientific  theories  and  principles  .  These  observations 
and  comments  do  not  claim  to  be  exhaustive  . 

a)The  expressions  "  Coniputcr-Aidcd  Learning"  , 
"Computer-Supported  Learning"  and  the  like,  point  to  a 
wide  scientific  area  ol  Education  where  the  information 
technology  participates  in  various  manners  and  degrees. 
We  do  not  intend  to  give  a  scientific  .^jluiion  to  these 
problems  of  computer-supported  education  but  cer- 
tainly we  have  to  face  an  interaction  among  different 
disciplines  concerning  behavioral  sciences,  decision- 
making, psychology,  computer  science.  And  we  cite 
only  some  examples. 

A  project  for  an  educational  system  must  always 
lake  into  account  the  organizational  environment  within 
which  this  project  is  to  be  carried  on.  Within  this 
environment  we  have  lo  lulfill  the  definition  of 
purposes  of  the  project,  the  cost- benefit  *>nalysis,thc 
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lime  interval  for  the  development,etc,  If  the 
environment  is  of  a  more  definite  administrative  nature 
like  thai  of  a  profit-oriented  company  or  that  of  a  slate 
agency,  we  should  not  undervalue  that  "learn- 
ing","communicaiion",  "education",  arc  "premises"  lo 
the  "decision-making"  of  that  organization,  and  not  the 
only  ones.  Sooner  or  later  we  shall  have  lo  lake  into 
account  other  premises  like  "authority",  "the  criterion  of 
efficiency"  ,  "loyalties  and  organizational  identifi- 
cations" eic.  (Simon, 1961).  In  other  words,  the 
purpose  of  an  educational  project  within  an 
organizational  environment  plays  a  role  of  premise  lo 
the  decision  making  (it  does  not  matter  whether  at  the 
instructional  Ievel,for  gaining  purposes,  or  at  the  higher 
indoctrination  level)  and  it  is  quite  difficult  lo 
disentangle  this  premise  from  the  other  ones  previously 
mentioned. 

Furthermore  let  us  remember  how  the  educational 
function  is  ever  more  acknowledged  in  modern 
organizations  and  lliis  function  is  often  carried  out  by  an 
organizational  unit  of  high  level  in  the  organizational 
line.  These  educational  projects  have  some  analogy 
with  the  Management  Information  Systems  as  regards 
the  variety  of  experts  who  participate  in  these  projects. 
Within  these  teams  we  find  the  same  premises  ,  above 
mentioned  ,  for  the  decision  making  concerning  the 
project. 

The  Slate  of  affairs  docs  not  vary  if  we  consider  an 
educaiionai  project  for  an  organiz.ation  like  a  school  or 
university.  A  class  of  students  or  a  sliiff  of  teachers 
participate  in  an  organization  where  it  is  difficult  to 
disentangle  those  elements  like  authority, 
communication,  efficiency,  etc.  as  previously  said. 

The  role  played  by  information  technology  in  this 
Slate  of  affairs  is  not  easy  to  define.  From  the  historical 
\mni  of  view,  this  technology  has  played  the  role  of 
powerful  change  in  many  areas  of  Science.  If  we 
observe  this  role  within  tin  educational  project ,  we  may 
observe  influences  of  viuious  nature. 

b)  information  technology  evokes  a  sense  of 
efficiency  potential,  of  improvement  of  the  educational 
system  and,  generally  speaking,  of  enhancing  the 
organizational  development.  One  might  say  that  the 
"quality"  of  the  educational  system  and  particularly  ilie 
"quality"  of  the  learning  processes  receive  this  benefit. 
The  conditions  that  allow  this  benefit  are  often  un- 
dervalued or  not  well  analyzed. 

The  theme  of  quality  of  products  ,  or  better  the 
theme  of  the  so-called  "total  quality"  has  received 
particular  attention  for  a  long  time  .  Let  us  remember 
some  pioneers  like  Deming  ,  Juran  and  Feighenbaum  , 
and  that  the  Total  Quality  Management  (TQM) 
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movement  has  evolved  into  a  whole  philosophy  of 
management  (Kleindorfer-Kunreuter-Shoemaker,1992) . 
This  philosophy  has  strong  links  with  the  Japanese 
economic  development. 

"How  do  you  make  quality  hapjxjn?"  Deining  asks 
(Anjani  Jain  J  992)  and  then  answers  this  question  as 
follows  .  The  enterprise  must ,  lie  says:  1)  CrciUe  new 
and  improved  products.  2)  Create  new  and  improved 
processes.  3)  Modify  and  improve  existing  products.  4) 
Modify  and  improve  existing  processes. 

In  our  investigation  we  arc  not  very  interested  in 
"products"  but  certainly  in  "processes",  and  among  these 
let  us  focus  upon  educational  processes.  Many  leader 
enterprises  have  paid  and  pay  much  attention  to  the 
educational  processes  (indoctrination,  instructional 
activities,  etc.)  .  Nevertheless,  it  does  not  seem  that 
particular  care  is  devoted  to  models,  tools,  etc.,  let  us 
say  by  a  statement  ,to  the  "how  s"  of  this  administrative 
learning  We  might  also  say  "  particular  care 

regarding  the  conditions  for  improving  quality  of 
learning  processes  ".  The  "how"  of  a  process  concerns 
the  "quality"  of  the  process. 

c)  Among  the  TQM  methodologies  we  find  the 
Quality  Improvement  Cycle  (QIC)  for  improving 
processes  and  products  (Kleindorfer,  1992) .  This  cycle 
may  be  explained  by  the  following  loop  :  1)  Select 
improvement  area.  2)  Identify  outputs  and  customers. 
3)  Detennine  customer  expectations.  4)Describc  current 
process.  5)  Focus  on  improvement  opportunities.  6) 
Determine  root  causes.  7)  Trial  and  implement  solution. 
8)  Hold  llie  gains.  1)  Select....  2) ....  It  is  not  difficult  to 
apply  this  cycle  to  the  improvement  of  learning 
processes.  As  an  example,  at  points  2)  and  3)  ,  replace 
the  word  "customer"  with  the  word  "individual"  who 
benefits  from  the  learning  process.  At  point  5)  the  word 
"  opportunities"  may  indicate  a  technological  support; 
....etc.  This  cycle  is  loop-like  in  that  administrative 
activities  have  a  repetitive  nature. 

d)  Points  1)  ,  4)  and  5)  of  ihe  above  mentioned 
QIC  ,  cannot  be  undervalued.  We  must  select  a  new 
organizational  area  ,  describe  the  learning  process  and 
project  a  new  one  1 

On  this  subject  let  us  mention  some  key  -attributes 
affecting  the  quality  of  learning  processes.  It  is 
interesting  to  note  how  for  each  of  these  attributes  one 
can  find  an  information  technological  support.  Let  us 
remember  a  short  list  of  these  .  1 )  Number  of  Iciirners  , 
level  of  conflict-cooperation  to  be  carried  on  .  Sec  the 
experiments  b)  and  c) .  The  use  of  a  local  area  network 
may  enhance,  even  if  not  necessarily,  this  kind  of 
learning  .  2)  Form  of  language  used  in  the  user 
interface  :  traditional  or  through  images,  colors,  or  even 


sounds  .  3)  Kind  of  events  flowing  .  In  experiment 
c)  there  is  an  alternating  of  elecu*onic  messages  and  hu- 
man messages  among  teams  of  learners  .  4)  More  or 
less  strong  reinforcement  of  the  message,  of  the 
communication  .  This  attribute  is  particularly  imporiiint 
in  experiments  like  d).  5)  Evaluation  of  learning  , 
more  or  less  explicit  and  specific.  See  experiment  c)  as 
an  example  of  not  explicit  evaluation,  and  experiment  d) 
as  an  example  of  explicit  and  specific  evaluation.  6) 
Form  of  knowledge  and  skill  to  be  reached  .  In 
experiment  a)  ,  the  integrated  and  process  knowledge 
was  'he  focus  of  the  model .  7)  Tlie  nature  of  concepts 
and  the  level  of  abstraction  ,  the  realism  of 
representations  .  See  experiment  a)  in  which  the 
simulation  of  processes  takes  into  account  their 
simultaneity,  the  analysis  of  tJie  system  arrives  at  the 
elementary  unit  of  products,  etc.  In  Computer  Science 
there  are  specific  methodologies  for  representing  dala  in 
order  to  create  data  bases  (see  entity-relationship 
model).  By  analogy  we  may  use  methodologies  for 
representing  objects  belonging  to  administrative 
processes  in  order  to  create  computer-based  educational 
systems  .  Languages  of  logic  programming  belong  to 
this  kind  of  considerations. 

The  above  detailed  list  of  attributes  affecting  the 
"quality  "  of  an  educational  process  is  obviously  not 
exhaustive  .  As  regards  the  "essence"  of  computing  in 
the  educational  process  ,  and  therefore  a  way  of 
considering  this  kind  of  attributes  for  their  educational 
quality  ,  see  Taylor,  1 991. 

We  have  exposed  this  non-exhaustive  list  of  ar- 
guments affecting  quality  of  computer-supported 
educational  projecLs  in  order  to  miike  clear  "how"  the 
technological  support  ,  both  iiard  and  soft,  may  enter 
into  the  design  of  these  projects. 

Problems  concerning  the  effectiveness  of  these 
computer-based  educational  projects  have  not  been 
faced  (for  a  study  concerning  effectiveness  evalaition 
of  ITS  see,  as  an  example,  Legree  and  Gillis  ,1991) . 
Even  if  this  kind  of  problems  is  important  from  an 
empirical  point  of  view  ,  we  believe  wc  have  implicitly 
demonsu-ated  how  these  computer-based  educational 
projects  are  strongly  connected  to  liie  purpose  which 
"molds"  the  system.  The  variety  of  sohnions  suggests  a 
different  approach  strictly  connected  to  the  design 
research. 


Conclusions 

When  we  have  to  deal  with  educational  projecLs  in 
which  information  technology  has  more  or  less  weight, 
we  discover    that  we  have  to  deal  with  artificial 
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phenomena  where  "  ihc  contingency  has  created  cloubLs 
as  to  whether  they  Tall  properly  within  the  compass  of 
science"  (H.A.Simon  ,i978,Prclace),  where  the  system 
is  molded  by  its  goals  or  pur|)oses  which  in  their  turn 
connect  or  link  Uiis  system  to  ius  environment . 

Our  main  attention  has  been  to  the  area  of  design 
research,  and  we  have  not  faced  the  problem  of 
whether  we  have  to  dctil  with  affairs  of  Science  or 
Art,  but  we  have  sjm[)ly  discussed  some  elemenus 
which  participate  in  the  linking  prcKOss  between  the 
goals  or  purposes  ,on  one  hand,  and  ,  on  the  other  ,  the 
organizational  environment  within  which  the 
educational  project  is  immersed.  In  doing  so  we  have 
not  undervalued  the  fact  that  our  mvestigations  on 
learning  fall  within  the  compass  ol  the  individual, 
within  the  compass  of  his  inner  being. 
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Object  Oriented  Music  Theory  Courseware 

Paul  E,  Dworak,  University  of  North  Texas 


This  showcase  demonstrates  to  the  viewer  the  advantages  of  developing  courseware  using  an 
object  oriented  language.  The  freshman  ear  training  text  used  is  Basic  Ear  Training  Skills, 
written  by  Robert  Ottman  and  this  author.  The  accompanying  courseware  was  written  in  C  +  + 
using  MacApp,  an  object  orienteo  development  system  for  Macintosh  applications. 

In  this  presentation,  the  viewer  will  see  how  the  object  oriented  environment  encourages  an 
intuitive  software  design  based  on  defining  the  courseware  in  terms  of  functional  units  that 
correspond  with  activities  actually  carried  out  by  students  in  completing  an  ear  training  exercise. 
This  presentation  will  demonstrate  objects  used  in  this  courseware.  Although  the  presentation 
will  not  explain  the  object  hierarchies,  it  will  give  the  viewer  a  basic  understanding  of  how  these 
objects  function  and  will  suggest  how  such  objects  might  be  used  in  courseware  developed  by 
the  viewer. 
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CREATING  COURSEWARE  CATALOGS:     THE  MARRIAGE  OF  DATABASE 
TECHNOLOGY  TO  CBI  SYSTEMS  — -  AN  INSIDER'S  VIEW 


Presented  by: 

Elizabeth  C.  Miller  and         Pamela  T.  Hess 

Senior  Consultant  Senlorr  Systems  Analyst 

Control  Data  Corporation  Control  Data  Corporation 


1.0  Introduction 

I .  1  Purpose 

Throughout  the  U.S.,  hundreds  of  people  in  the  commercial; 
government,  and  academic  communities  have  been  developing 
CAI  courseware  for  more  than  20  years.     Because  people  were 
not  aware  of  previously  developed  courseware  these  efforts 
have  been  duplica-  3d  time  and  time  again.     In  an  effort  to 
eliminate  this  duplication,  the  Joint  Committee  on  Computer- 
Based  Instruction  (JCCBI)  members  voted  to  collect, 
organize,  and  catalog  all  CYBER  based  courseware.     As  a 
result  of  this,  the  JCCBI  teamed  with  the  Training  and 
Performance  Data  Center  (TPDC)  to  produce  a  catalog  composed 
of  over  3400  entries  which  represent  more  than  5000  hours  of 
CAI  instruction.     This  CATALOG  is  not  only  valuable  to  the 
JCCBI  but  also  to  other  courseware  developers  who  can  access 
it  either  by  hard  copy  or  interactively  in  real  time, 

II.  I    Project  Scope 

The  CATALOG  project  contains  the  following  information: 

m        Rationale  for  developing  the  Joint  Service  CYBIS 

Courseware  Catalog 
B        Major  tasks/milestones  performed  during  the  project 

■  Methodology  used  in  compiling  the  Department  of  Defense 
(DoD)  courseware  information 

■  Description  of  the  catalog  organization  and  associated 
data  fields 

■  Planning  and  Development  of  an  automated  CYBER 
application  (CBT  lesson)  which  would  allow  JCBIS  users 
to  access  the  CATALOG  in  an  orderly  fashion  —  the 
CATALOG  operates  like  the  Control  Data  Corporation 
SHIFT-F  CATALOG • 

■  Documentation  of  the  CATALOG  database  and  application 
to  accommodate  future  modifications 

■  Documentation  of  courses  that  are  available  for 
American  Council  on  Education  (ACE)  college  credit 
recommendations  are  also  called  out  in  the  CATALOG, 
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The  basic  approach  taken  for  this  project  w^s  to  produce  a 
CATALOG  database  for  the  long-term,  update  the  data  fields 
based  on  a  user  survey,  develop  the  CATALOG  application,  and 
establish  the  CATALOG  on  the  JCBIS  computer  systems. 

The  following  activities  were  conducted  during  this  project: 

■  Kick-off  Meeting  and  JCCBI  Briefing 
m        Database  Design  and  Convention 

m       First  Questionnaire  Development  and  Circulation 
B        First  Database  Update  and  CYBER  Upload 

■  On-line  CATALOG  Modifications 

B        Final  On-line  CATALOG  Modification  &  Documentation 
1.2    Catalog  Design 

The  major  portion  of  this  report  is  the  CATALOG  of  Joint 
Service  Courseware.     This  portion  contains  computer-based 
instruction  identified  into  major  training/ education 
categories.     These  categories  include: 

B  Aviation 

n  CYBIS  Utilities 

a  Data  Processing 

n  Engineering 

B  Foreign  Language 

B  General  Education 

B  Job  Skills  Education  Program  (JSEP) 

B  Management 

B  Medical 

B  Military 

B  Technical  Training 

1,2,1         Catalog  Category  -  Aviation 

The  Aviation  Training  Category  in  the  CATALOG  is  composed  of 
the  following  sub-topics: 

B  Airways  Facilities  Maintenance 

B  CARDION  DMEFA-9783 

B  Communications  Maintenance 

B  Data  Communications 

B  Flasher  System  FA-9989 

B  Helicopters 

B  High  Intensity  ALS 

B  ILS 

B  Maintenance 

B  Navigation  Aids 

B  Pilot  Training 

B  RADAR 

B  VASI 
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U  VORTAC 

m         visual  Landing  Aids 


1.2.2 


Catalog  Category  —  CYBIS  Utilities 


The  PLATO  Utilities  Category  is  divided  into  the  following 
sub-topics : 

■  Character  Sets 

■  Datasets 

■  New  User  Training 
B  Notefiles 

■  Test/Quiz  Generators 
a  Utilities 


1.2.3 


Catalog  Category  —  Data  Processing 


The  Data  Processing  Category  in  the  Joint  Service  Catalog 
composed  of  the  following  sub-topics: 

Computer  Hardware 
Computer  Literacy 
Computer  Software 
Data  Processing 
Programming  Languages 
Programming  Languages: 
Programming  Languages: 
Programming  Languages : 
Programming  Languages: 
Programming  Languages: 


Ada 
C 

COBOL 
FORTRAN  7  7 
TUTOR 


1.2-4 


Catalog  Category  —  Engineering 


The  Engineering  Category  in  the  Joint  Service  Catalog  is 
composed  of  the  following  sub-topics  defined  by  USAES: 

Analytical  Bridge  Design 
Bridge  Design  Fundamentals- 


Concrete 


Construction  Management 
Drainage  Design  / 
Drainage  Facilities  ■ 
Engineering  Miscellaneous 
Equipment  Utilization 
Geology 

Hasty  Bridge  Design 
ICAM 

Materials  Quality 
Road  Construction 
Road  Design 

Road  Design  Simulation 
Roads  and  Airfields 
Utilities 
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1.2 .5  Catalog  Category  —  Foreign  Language 

The  Foreign  Language  Category  in  the  Joint  Service  Catalog 
is  composed  of  the  following  sub-topics: 

■  Arabic 
a  Chinese 

■  Czech 
B  Egyptian 

■  French 

■  German 

■  Greek 

■  Italian 

■  Japanese 

■  Korean 
m  Latin 

■  Miscellaneous 

■  Polish 

■  Portuguese 

■  Russian 

■  Serbo-Croatian 

■  Spanish 

■  Syrian 

■  Turkish 
B  Vietnamese 

1.2.6  Catalog  Category  —  General  Education 

The  General  Education  Category  in  the  Joint  Service  Catalog 
is  composed  of  the  following  sub-topics: 

B  AFCT/ASVAB 

B  BNCOC 

B  BSEP 

B  Broadcasting 

B  CLEP 

B  Career  Development 

B  Education  Services 

'B  GED 

B  GRE  Exam 

B  GT 

B  Journalism 

B  Learning  Center  Staff  Training 

B  Mathematics 

B  Photography 

B  Statistics 

B  Training  and  Development 

B  Vocabulary 


1,2*7  Catalog  Category  —  JSEP 

The  Job  Skills  Education  Program  (JSEP)  is  an  on-duty, 
computer-»based  curriculum  that  provides  job-related 
instruction.     JSEP  lessons  develop  the  soldier's  basic 
education  skills  couched  in  a  relevant  Army  context.  These 
basic  skills  were  selected  for  inclusion  because  they  are 
much  more  likely  to  c.-ieralize  across  Military  Occupational 
Specialities  (MOS)  a.     job  tasks.     The  lessons  in  the  JSEP 
curriculum  are  clustered  according  to  the  requirements  of 
the  94  most  populous  MOS.     The  goal  of  the  JSEP  effort  is  to 
improve  soldier's  general  skills,  thus  enabling  them  to  meet 
the  requirements  of  their  jobs  more  completely,  and  to  learn 
new  tasks  more  efficiently. 

Under  the  JSEP  Category  the  following  sub-topics  have  been 
defined  by  Florida  State  University: 

■  Addition  and  Subtraction 

■  Algebra 

■  Angles  and  Triangles 
n  Degree  Measures 

■  Editing 

Bi  Flow  Charts 

■  Forms 

■  Fractions/Decimals 

■  Gage  Measures 

■  Geometry 

■  Graphing  in  the  Coordinate  Plane 

■  Illustrations 
B  JSEP  Graphics 

■  Learning  Management 
B  Learning  Strategies 

B  Linear,  Weight,  and  Volume  Measures 

B  Lines 

B  Multiplication  and  Division 

B  Note-Taking 

B  Numbering  and  Counting 

B  Outlining  (Topic  and  Sentence) 

B  Planes 

B  Precautions 

B  Procedural  Directions 

B  Process  Combinations 

B  Recognition 

B  Reference  Skills 

B  Report  Writing 

B  Schematics 

B  Solids 

B  Spatial 

B  Supplemental  Topics 

B  Tables/Charts 

B  Terminology 
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1.2.8 


Time-Telling  Measures 

Trigonometry 

Vocabulary 

Catalog  Category  —  Management 


The  Management  Category  in  the  Joint  Service  Catalog 
composed  of  the  following  sub-topics: 

B  EEO/AA 

B  Finance 

B  Managerial  Training 

B  Money  Management 

B  Office  Administration 

B  Production  Control  Management 

B  Safety 

1.2.9  Catalog  Category  —  Medicine 

The  Medicine  Category  in  the  Joint  Service  Catalog  is 
composed  of  the  following  sub-topics: 


Administration 
Anatomy  and  Physiology 
Behavior /Psychiatry 
Cardiology 
Clinical  Medicine 
Community  Health 
Course  Management 
Dental  Science 
Dermatology 
Emergency  Medicine 
Endocrinology 
Ethics 
Hygiene 

Infectious  Disease 

Laboratory  Technology 

Medical  Readiness 

Medical  Terminology 

Medicine 

Mental  Health 

Miscellaneous 

Neurology 

Nursing 

Nutrition 

Obstetrics /Gynecology 
Oncology 

Optometry/Ophthalmology 

Orthopedics 

Patient  Records 

Pediatrics 

Pharmacology 
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■  Physical  Therapy 

■  Public  Health 

■  Pulmonary  Medicine 

■  Radiology 

■  Safety 

■  Security 
m  Surgery 

■  Tests/Quizzes 

■  Typing 
n  Urology 

B  Veterinary  Medicine 

m  Wellness 

1.2.10        Catalog  Category  —  Military 

The  Military  Catalog  in  the  Joint  Service  Catalog  is 
composed  of  the  follov/ing  sub-topics: 

m  Administration 

n  Armor 

■  Drill  and  Ceremonies 

■  Drug/Alcohol  Abuse 

■  First  Aid 

■  HAWK 

n  International  Relations 

■  Leadership/Command 

■  Logistics 

■  Map  Reading 
n  Military 

B  Military  Terminology 

B  NBC 

fl  Promotion 

B  Security 

B  Tactics 

B  Technical  Orders 

a  TOW 


1.2*11        CATALOG  Category  —  Technical  Training 

The  Technical  Training  Category  in  the  JCCBI  Catalog  is 
composed  of  the  following  sub-topics: 

B  Automotive  Mechanics 

B         Basic  Electronics 
B  Hydraulics 
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Examples  of  Instructional  Empowerment  Using  ToolBook 

Carol  Dwyer  and  Morris  Weinstock,  Perm  State  University 

ToolBook  by  Asymetrix  Corporation  is  an  interactive  multimedia  development  tool  for  the  IBM 
Windows  environment.  ToolBook  combines  buttons,  text,  and  graphics  on  the  "pages"  of 
interactive  "books". 

Faculty  from  Penn  State  University  are  using  ToolBook  to  develop  instructional  software  and 
interactive  classroom  presentations  with  help  from  the  staff  of  the  Teaching  and  Learning 
Technologies  Group»  Products  include  "SPECS:  Selecting  Production  Equipment  with  Computer 
Simulations,"  developed  with  a  faculty  member  from  Hotel,  Restaurant,  and  Institutional 
Management,  and  "Computerized  Glossing  of  Literacy  Texts  Written  in  French,"  developed  with 
faculty  from  the  College  of  Liberal  Arts.  In  addition,  with  limited  assistance  from  our  staff, 
seme  faculty  are  creating  interactive  "electronic  transparencies"  for  large  class  lectures  in 
economics  and  statistics. 

These  products  will  be  demonstrated,  while  discussing  instructional  problems,  i^^structional 
solutions,  as  well  as  design  and  development  issues.  Participants  will  see  high-quality  design 
of  screens,  interactions,  and  pedagogical  strategies.  They  will  have  the  opportunity  to  visually 
inspect  how  instructional  problems  were  solved  through  the  integrated  use  of  icons,  animation, 
color  graphics,  etc. 

We  will  begin  with  a  brief  overview  of  the  mission  and  activities  of  our  Teaching  and  Learning 
Technologies  Group,  how  we  solicit  proposals  for  development  projects  from  faculty,  and  how 
we  work  as  a  team  to  carry  out  the  project.  We  will  also  engage  participants  in  discussion  about 
design  and  development  issues  and  solutions. 


The  Student  of  Socrates  Meets  the  Students  of  Kentucky  PLATO(R) 
and  Western  Kentucky  University 

Kenneth  L.  Modesitt,  Western  Kentucky  University 

The  problem:  how  to  introduce  and  maintain  effective  use  of  computer-based  learning  on  a 
significant  scale  in  academia?  The  rapidly  growing  number  of  alternative  hardware, 
communication,  software,  courseware  and  authoring  systems  can  make  for  considerable 
confusion.  Therefore,  wise  planning  involves: 

-key  personnel  at  both  faculty  and  administrative  levels, 

-active  participation  of  a  willing  and  highly-qualified  courseware  corporation, 

-the  choice  of  a  "guaranteed  success"  in  the  formative  stages, 

-the  involvement  of  respected  "champions",  and 

-considerable  ingenuity  in  determining  financial  feasibility,  esp  now! 
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The  solution  involves  persistence  in  pursuing  a  plan  to  help  students  learn  better  so  that  they  can 
become  productive,  self-sufficient  and  contributing  members  of  society  for  the  rest  of  their  lives. 
The  ingredients  are:  students,  faculty,  administration,  tools,  financial  resources,  and  long-term 
vision  and  commitment,  and  a  desire/ability  to  work  with  industry.  Most  universities  have  the 
first  five  ingredients,  but  are  lacking  the  latter  three. 

Vignettes  of  the  personal  tales  of  traditional,  under-prepared  and  non-traditional  students  will 
be  interwoven  in  the  presentation  with  recommending  long-term  strategies  for  success  of 
significant  reform  in  the  university.  The  participant  will  be  able  to  apply  the  knowledge  that 
such  planning  can  result  in  success,  and  will  have  additional  evidence  when  promoting  similar 
ventures  at  his  own  institution. 


Reality-Based  Strategic  Planning  for  Computing  in  University-Based  Schools/Departments 

Ramona  Nelson,  University  of  Pittsburgh  Medicul  Center; 
Pamela  Hepple,  University  of  Pittsburgh  School  of  Nursing 

Higher  education  institutions  are  moving  from  a  centralized  main-frame  environment  to  a 
distributed  PC  based  environment.  At  the  same  time  they  are  caught  between  tight  budgets  and 
rapid  technological  change.  As  a  consequence  there  is  a  proliferation  of  approaches  to 
hardware,  software,  and  computer  use. 

This  presentation  explained  how  to  develop  a  strategic  planning  process  for  computing  in  a 
school  or  department  within  a  university.  Using  both  theory  and  experience  the  presenters 
described  a  variety  of  realistic,  effective  approaches.  An  outline  of  the  topics  and  issues  covered 
follows: 

Establish  the  planning  team. 

-Who  should  be  involved? 
Identifying  the  purpose  and  scope  of  the  planning  process. 

-Are  all  constituents  and  their  activities  addressed? 
Collect  pertinent  data. 

-What  are  the  sources  of  data? 
-What  approaches  to  data  collection  are  useful? 
Analyze  the  data. 

-How  are  past  trends  related  to  planning  for  the  future? 
Design  an  ongoing  planning  process  and  write  the  initial  plan. 

-How  does  one  design  and  organize  a  planning  process? 
Market  the  plan  and  the  planning  process  to  key  decision  makers. 

-How  can  a  vision  be  connected  to  goals? 
Implement  the  plan  and  the  process. 

-How  can  implementation  be  flexible,  yet  effective? 
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Faculty  Development  and  Support  for  Development  and  Utilization 
of  Computer-Based  Instruction:  Experiences  along  the  Continuum 

Floyd  K.  (Rusty)  Russell,  Stan  Cohen,  and  Anne  Nardi,  West  Virginia  University; 
Carol  Dwyer,  Pennsylvania  State  University, 
and  Charlotte  Scherer,  Bowling  Green  State  University 

Panel  members  will  discuss  how  three  different  institutions  are  providing  faculty  development 
opportunities  for  learning  how  to  develop  and  use  interactive  technologies  for  instruction.  These 
efforts  represent  stages  along  a  continuum  of  start-up  efforts  to  advances  support  activities. 

The  session  includes  information  about  the  CACHE  incentive  program  in  the  West  Virginia 
University  (WVU)  College  of  Arts  and  Sciences  and  information  about  faculty  development 
workshops  sponsored  by  the  Office  of  Faculty  Development  in  conjunction  with  the  WVU 
Computer-Based  Learning  Forum.  Content  from  the  Penn  State  University  segment  includes 
information  about  the  Teaching  and  Learning  Technologies  Group  and  the  competitive  awards 
program  for  supporting  faculty  in  the  development  of  computer  based  learning  materials. 
Material  about  Bowling  Green  State  University  includes  information  about  workshops,  informal 
support  in  computer  laboratories,  and  a  long  range  plan  for  computerizing  the  College  of 
Education  and  Allied  Professionals  by  upgrading  laboratories,  providing  computers  in  faculty 
offices,  and  networking  the  college.  The  discussion  also  includes  plans  for  a  high  technology 
classroom  building  and  its  role  as  an  incentive  for  faculty  development  of  computer  based 
instruction. 

Focus  areas  for  the  session  include  1)  types  of  workshops  and  their  content,  2)  incentive 
programs  for  encouraging  faculty  development  of  materials,  3)  design  and  production  support, 
and  4)  vendor  support  for  development  of  materials. 


Conmiunicating  Knowledge  about  LANs 

Donald  Scherer,  Bowling  Green  State  University 

This  presentation  discusses  three  central  problems  of  communication  in  the  installing  of  .an 
AppleTalk  network  in  an  academic  department.  The  problems  are  the  lack  of  a  shared 
vocabulary  between  people  in  computer  services  and  people  in  academic  departments,  the 
fragmentation  of  knowledge  among  people  in  computer  services  and  differing  computer  interests 
and  knowledge  among  people  in  academic  departments.  The  presentation  proposes  procedures 
for  minimizing  each  of  these  problems. 
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Multimedia  Lectures:  A  Comparison  of  Some  Macintosh  Alternatives 

Eileen  Schroeder,  Penn  State  University 


New  computer  programs  make  it  easier  for  instructors  to  recreate,  restructure,  and  reinvent  their 
lectures  using  computers  incorporating  graphics,  text,  and  various  types  of  media.  Each 
program  has  its  own  special  features  but  all  are  intended  to  simplify  the  process  and  remove  the 
need  for  extensive  programming  skills.  This  session  will  review  some  of  the  options  available 
to  faculty  members  desiring  to  use  the  computer  in  lecture  settings  through  an  examination  of 
the  varying  needs  and  abilities  of  faculty  members,  development  of  criteria  for  selecting  tools 
to  create  multimedia  lectures  on  a  Macintosh  platform,  review  of  several  currently  available 
alternatives,  and  comparison  of  sample  screens  from  comparable  lectures  using  each  of  the  tools. 
Programs  such  as  Mediatext,  Multimedia  Annotator,  Resource  Navigator  II  and  HyperCard  with 
accompanying  toolkits  were  reviewed  for  this  project.  Each  program  was  examined  for  its  ease 
of  use,  ability  to  access  other  applications,  ease  of  accessing  multimedia  such  as  CD  players  and 
videodisc  players,  capability  for  handling  a  variety  of  graphics,  audio,  and  text,  use  of  digitized 
video,  programming  expertise  required,  flexibility,  branching  capabilities,  adaptation  for 
individual  student  use,  and  cost.  A  prescriptive  matrix  matching  the  programs  examined  with 
faculty  needs  was  created. 
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The  Design,  Implementation,  and  Management  of  Lehigh 
University's  International  Multimedia  Resource  Center 


Blair  R.  Bernhardt 
Lehigh  University 


Abstract 

After  an  extended  period  of  planning,  Lehigh  University  recently 
opened  its  International  Multimedia  Resource  Center  (IMRC) .  The 
IMRC  consists  of  a  multimedia-based  microcomputer  classroom  for 
teaching  foreign  languages  and  world  culture;  the  World  View" 
Room,   in  which  international  satellite  transmissions  may  be 
viewed;  and,  the  head-end  room  in  which  cable  television  signals 
are  mixed  with  satellite  transmissions  as  well  as  video  from 
VCR's  and  videodiscs,  and  then  placed  onto  a  building-wide  cable 
plant.     All  classrooms  in  the  building  housing  the  IMRC  contain 
at  least  a  VCR  and  overhead  monitors  while  all  offices  contain 
cable  television  jacks.     It  is  the  intent  of  this  paper  to 
describe  the  design,   implementation,  and  management  of  the  IMRC. 


overview 

The  Lehigh  University  Department  of  Modern  Foreign  Languages 
and  Literature  (MFL) ,  in  conjunction  v;ith  other  departments  on 
campus,  had  been  working  on  the  design  of  the  IMRC  for 
approximately  two  years  when  they  requested  assistance  from  the 
Lehigh  University  Computing  Center  (LUCC)  and  the  Physical 
Planning  Office.     At  that  point,   funding  for  the  IMRC  had  already 
been  established  and  it  had  already  been  decided  that. IBM 
microcomputers  would  be  used  for  the  IMRC.     It  appeared,  at  least 
superficially,  that  the  only  thing  left  for  LUCC  and  Planning  to 
do  was  to  help  MFL  implement  this  plan.     Now,  after  over  a  year 
of  involvement,  the  IMRC  is  finally  taking  shape. 


The  Design  and  Implementation  of  the  IMRC 

Broadband  versus  Baseband 

After  consultation  with  other  Lehigh  departments,  as  well  as 
with  external  consultants,  the  original  MFL  plan  called  for 
running  individual  strands  of  baseband  cabling  to  all  IMRC 
microcomputers  as  well  as  to  all  classrooms.     This  approach  would 
have  required  additional  wiring  from  each  microcomputer,  as  well 
as  from  each  classroom,  back  to  a  remote  router  (to  change  the 
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channel)   in  the  head-end  room.     It  was  felt  that  this  approach 
would  result  in  a  better  quality  of  video  to  these  locations  as 
signals  from  devices  such  as  satellite  receivers  could  be  sent 
out  directly  without  the  degradation  resulting  from  having  to 
modulate  the  signal  to  put  it  on  to  a  broadband  cable  and  then 
demodulate  the  signal  to  get  it  off  of  a  broadband  cable. 

Once  LUCC  became  involved  and  examined  the  initial  plan,  it 
was  noted  that  each  of  the  microcomputers  was  to  also  be 
connected  to  a  VCR.     A  number  of  VCR's  would  be  placed  in  the 
head--end  room  and  accessed  from  classrooms  through  the  baseband 
cable.     The  VCR's  would  be  controlled  at  the  head-end  room  either 
manually  (by  calling  up  the  on-duty  operator  on  the  phone) ,  or 
via  remote  controls  (hooked  up  to  even  more  wiring) .  After 
consulting  the  local  cable  television  company,  LUCC  was  assured 
that  with  the  use  of  commercial  quality  modulators  any 
degradation  to  the  signal  to  place  it  on  a  broadband  cable  and 
then  remove  it  from  the  broadband  cable  would  not  be  noticeable. 

With  these  additional  facts,  a  decision  was  made  to  run 
broadband  cable  throughout  the  IMRC  and  throughout  the  building. 
This  decision  greatly  simplified  the  cabling  plan  and  also 
allowed  the  use  of  inexpensive  television  sets  for  video  display 
instead  of  using  more  expensive  video  monitors.     Placing  VCR's  in 
classrooms  provided  the  flexibility  required  for  viewing 
prerecorded  tapes. 


Domestic  versus  International  satellite  Receivers 

The  initial  plan  for  the  IMRC  called  for  the  installation  of 
two  satellite  receivers;  one  dish  for  domestic  reception,  and  one 
dish  for  international  reception.     Once  it  was  determined  that 
the  roof  of  the  building  was  structurally  sound  enough  to  hold 
it,  the  domestic  dish  was  ordered  and  installed  with  no  problem. 
The  international  dish  was  a  totally  different  situation. 

One  of  the  reasons  for  getting  an  international  dish  was  for 
reception  of  signals  from  the  Russian  satellites  Gorizont  7  and 
Gorizont  12;  both  of  which  are  low  to  the  horizon  from  Lehigh's 
geographic  position,  and  both  of  which  are  in  less  than  stable 
orbits.     While  people  familiar  with  domestic  satellite  dish 
installation  are  relatively  easy  to  find,  people  familiar  with 
international  satellite  dish  installation  are  much  harder  to 
find.  This  made  it  difficult  to  determine  whether  signals  from 
these  satellites  could  even  be  received  at  Lehigh.     While  it  was 
somewhat  assumed  that  these  signals  could  be  received,  it  would 
require  a  rather  large  receiver  dish  to  do  so.     As  this  dish 
would  be  significantly  larger  and  heavier  than  the  domestic  dish, 
and  would  also  be  much  more  prone  to  wind  stress,  it  was 
questionable  as  to  whether  this  dish  could  even  be  placed  on  the 
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roof  of  the  building  containing  the  IMRC. 

Lehigh's  campus  basically  straddles  a  mountain  (at  least  by 
eastern  Pennsylvania  standards) .     One  alternative  which  was 
considered  for  the  international  dish  was  to  place  it  on  top  of 
the  mountain  and  then  send  the  received  signals  down  to  the  main 
campus  through  either  microwave  transmission  or  via  fiber  optic 
cable.     Both  of  these  alternatives  v;ere  rejected  since  both  would 
have  significantly  increased  the  cost  of  the  international  dish. 
As  a  stopgap  measure   (at  least  until  additional  funding  was 
secured) ,   it  was  decided  to  subscribe  to  SCCLA.      [SCOLA,  the 
Satellite  Communication  for  Learning  network,   is  a  service  which 
receives  foreign  satellite  transmissions  (and  video  tapes)  and 
retransmits  them  for  reception  via  domestic  satellite  dishes.] 


Design  of  the  Multimedia-"Based  Microcompxiter  Classroom 

The  basic  design  of  the  multimedia-based  microcomputer 
classroom  was  primarily  a  function  of  the  triangular  room  in 
which  it  was  to  be  placed.     The  microcomputers  and  associated 
peripherals  were  placed  along  two  of  the  three  walls  with  a 
doorway  and  a  window  (through  which  headphones  and  tapes  could  be 
checked  out)   into  the  head-end  room  on  the  third  wall.  Cabling 
from  the  head-end  room  was  not  a  problem. 

Computer  Hardware  and  Software 

The  initial  IMRC  plan  called  for  the  use  of  microcomputers 
from  IBM.     IBM  had  already  provided  funding  for  the  development 
of  foreign  language  software  to  a  number  of  institutions  through 
its  Foreign  Language  Consortium.     A  stated  goal  of  this 
consortium  was  to  enhance  the  availability  of  foreign  language 
software,  and  it  was  the  hope  of  MFL  to  be  able  to  utilize  this 
software  while  its  faculty  members  were  in  the  process  of 
developing  additional  software. 

Around  the  time  that  the  specific  computer  hardware  and 
software  requirements  were  being  finalized,  IBM  announced  a  new 
multimedia  hardware  and  software  bundle  called  the  Advanced 
Academic  System  2.0  or  AAcS.     This  bundle  consisted  of  a  computer 
with  an  audio  card,  a  video  card,  a  CD-ROM  drive,  a  hand  scanner, 
and  a  full  contingent  of  software  to  make  these  components  work 
together  and  provide  the  tools  for  the  development  of  computer- 
aided  instruction.     With  the  exception  that  the  computers 
included  in  these  bundles  were  slightly  slower  than  those  which 
had  been  planned,  these  systems  met  or  exceeded  all  of  the 
finalized  requirements  at  a  cost  significantly  less  than 
anticipated.     As  cost  had  become  a  factor,  fifteen  of  these 
bundles  were  purchased. 
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As  is  often  the  case  when  purchasing  relatively  new 
technology,  a  number  of  problems  were  encountered  with  these 
bundles.     While  institutions  which  were  already  in  IBM's  Foreign 
Language  Consortium  were  supposedly  using  the  same  equipment, 
these  institutions  were  actually  using  the  prior  releases  of  most 
of  the  hardware  including  the  audio  card,  the  video  card,  and  the 
CD-ROM  drive.     To  make  matters  even  more  interesting,   in  some 
instances  the  provided  software  did  not  work  with  the  provided 
hardware.     For  example,  while  the  software  provided  an  option  for 
digitizing  CD-Audio,   and  while  the  CD-ROM  drive  was  capable  of 
playing  CD-Audio,  the  software  would  simply  go  through  the 
motions  of  digitizing  and  would  result  in  a  rather  huge  file 
filled  with  absolutely  nothing  of  value.     Upon  taking  apart  the 
machines,   it  was  determined  that  this  resulted  from  the  ^ack  of 
an  internal  connection  between  the  CD-ROM  drive  and  the  audio 
card. 

Another  problem  resulting  from  these  bundles  was  in  that  all 
of  the  software  had  been  pre-installed  on  the  hard  drives  and 
some  of  it  was  not  supplied  at  all  on  diskettes.     While  this  may 
have  been  perfectly  acceptable  for  stand-alone  systems,  to 
protect  these  systems  from  student  abuse  they  were  to  be  placed 
on  a  local  area  network.     Even  though,  with  effort,  the  software 
probably  could  be  installed  on  a  LAN,  some  of  the  pivotal  pieces 
of  softv/are  required  write-access  to  things  such  as  device 
drivers.     Since  students  do  not  have  write-access  to  software  on 
any  LUCC  LAN,  by  pJacing  this  software  on  a  LAN  the  flexibility 
of  the  software  would  be  limited  in  ah  unacceptable  manner. 

Laserdisc  Players 

Under  normal  circumstances,  given  the  hardware  and  software 
which  is  to  be  used,  picking  a  laserdisc  player  is  a  rather  easy 
task;  simply  check  which  players  are  supported  by  the  hardware 
and  software.     Unfortunately,   just  as  mid-level  players  were  to 
be  ordered,  the  producer  of  the  players  announced  a  change  in 
their  product  line,  effectively  removing  the  middle  of  their  line 
and  leaving  nothing  but  a  low-end  model  and  a  high-end  model. 
Obtaining  a  source  for  fifteen  players  of  a  discontinued  model 
was  not  feasible.     In  discussions  with  representatives  of  the 
company,   it  was  determined  that  a  new  middle  of  the  line  model 
was  to  be  announced  shortly  thereafter  which,  along  with  the 
video  card  which  was  to  be  used,  would  provide  most  of  the 
functionality  of  their  high-end  player.     At  that  point,   an  order 
for  the  new  players  was  placed.     Six  months  later,  the  players 
were  received, 

VCR's 

In  addition  to  a  laserdisc  player,  each  microcomputer  also 
wa3  connected  to  a  VCR.     The  VCR  was  to  have  two  basic  functions: 
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to  provide  stand-alone  playback  for  prerecorded  tapes,  and  to 
function  as  a  tuner  to  select  channels  from  the  broadband  cable. 
As  the  video  card  was  capable  of  accepting  up  to  three  different 
video  signals  at  a  time,   it  was  assumed  that  it  would  be  a 
relatively  simple  task  to  put  together  a  simple  program  which 
would  allow  the  user  to  select  either  the  laserdisc  player  or  the 
VCR  for  the  video  input.     Unfortunately,  the  majority  of  the 
control  program  for  the  video  card  had  not  been  included  in  the 
AAcS  bundle.     ^fter  discussions  with  IBM,  a  full  copy  of  the 
control  program  was  received  which  contained  the  necessary 
drivers  for  the  development  of  a  video  source  selection  program. 

While  the  vidp.n  source  selection  program  allowed  video  from 
a  VCR  to  be  displayed  on  the  screen,   it  still  required  each  VCR 
to  be  operated  manually.     A  number  of  computer-controllable  VCR*s 
were  purchased  for  developmental  work,  however,  the  cost  of  these 
devices  was  sufficiently  high  that  little  consideration  was  given 
to  placing  them  at  each  of  the  microcomputer  stations. 

Microphones  and  Headphones 

As  audio  experts  are  not  typically  found  within  foreign 
language  departments,  computing  centers,   or  physical  planning 
offices,   it  was  basically  assumed  that  it  would  be  safe  to 
purchase  these  items  with  some  basic  specifications  from 
catalogs.     By  purchasing  moderately  priced  professional  quality 
microphones  and  headphones  the  supplier  also  assumed  everything 
would  be  fine.     This  was  not  the  case. 

Unfortunately,  the  headphones  turned  out  to  be  not  quite 
professional  quality,  but  rather  medium-end  consumer  models. 
While  the  sound  that  the  headphones  delivered  was  acceptable,  it 
was  obvious  that  they  would  not  hold  up  to  the  rigors  of  student 
use.     After  testing  a  nun.ber  of  other  headphones,   it  was  decided 
that  the  best  alternative  was  to  return  the  expent  ive  headphones 
and  buy  a  larger  quantity  of  very  inexpensive  headphones  instead. 
Also,  any  students  who  wish  to  are  encouraged  to  simply  supply 
their  own  personal-cassette  player  type  headphones. 

Microphones  created  even  more  interesting  problems.  First, 
professional  model  microphones  do  not  come  with  connectors 
attached  but  simply  with  wires  hanging  out  of  the  end.     It  is 
assumed  that  whoever  purchases  these  models  will  have  the 
technical  expertise  required  to  wire  them  and  make  them  work. 
Next,  there  is  the  problem  of  the  sound  level  which  these 
microphones  deliver.     While  the  sound  level  is  fine  for  the 
devices  to  which  these  low-impedance  microphones  are  typically 
connected,   it  was  insufficient  for  ui>e  with  the  audio  cards  which 
were  expecting  input  from  high-impedance  microphones.  This 
problem  was  resolved  by  placing  an  impedance  matcher  (with  its 
associated  connectors  and  wires)  between  che  microphone  and  the 
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audio  card.     Upon  further  testing,   it  was  determined  that  this 
problem  could  have  been  resolved  even  easier  by  simply  purchasing 
cheap  replacement  microphones  (which  are  high-impedance)   from  any 
electronics  store . 

For  faculty  development  use,  the  MFL  department  also  wanted 
a  number  of  headsets  which  combined  headphones  and  microphones 
into  one  unit.     These  devices  were  also  ordered  from  a  catalog, 
also  came  without  any  connectors  (but  twice  as  many  wires  for  the 
two  devices) ,  and  also  had  the  same  impedance  problem  with  the 
microphones.     As  if  this  wasn't  enough,  the  instructions  for 
wiring  these  headsets  were  written  only  in  German. 


Design  of  the  World  View  Room  and  Classrooms 

The  building  in  which  the  IMRC  resides  is  of  the  somewhat 
modern  design  which  was  popular  in  the  1960 's.     Form  took 
precedence  over  functionality.     Hence,   like  the  microcomputer 
classroom,  the  World  View  Room  and  many  of  the  classrooms  are 
triangular  in  shape.     For  the  World  View  Room,  this  shape 
actually  enhances  the  design  with  a  large  screen  projection 
television  as  the  focal  point  in  one  corner  of  the  room  and 
chairs  arra  iged  radiating  out  from  it.     Another  corner  of  the 
room  contains  shortwave  radio  monitors  while  the  third  corner 
contains  foreign  language  reading  materials.     World  maps  line  the 
walls.     The  television  is  usually  tuned  to  SCOLA  which  provides 
broadcasts,  either  live  or  slightly  delayed,   from  around  the 
world. 

The  classrooms  are  supplied  with  either  one  or  two 
televisions  suspended  from  the  ceilings  (depending  on  the  size  of 
the  room) ,  and  a  VCR,     As  all  of  the  classrooms  are  also  supplied 
with  connections  to  the  broadband  cable,  this  setup  allows 
classes  to  utilize  either  live  broadcasts  from  SCOLA,  C-SPAN,  or 
CNN,  or  any  prerecorded  video  tapes.     One  of  the  larger 
classrooms  is  supplied  with  a  projection  system  connected  to  the 
same  type  of  multimedia-based  computer  system  found  within  the 
computer  classroom. 


The  Management  of  the  IMRC 

Daily  operation 

The  mission  of  the  IMRC  is  to  promote  cultural  awareness  and 
to  develop  software  for  foreign  language  instruction*     To  achieve 
these  objectives,   a  position  of  Director  of  the  IMRC  was 
established.     The  director  handles  the  daily  operations  of  the 
IMRC,   including  things  such  as  programming  the  satellite  dish  and 
VCR*s,  scheduling  of  IMRC  resources,  and  ensuring  that  all  IMRC 
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resources  are  functioning.     It  is  the  role  of  the  director  to  not 
only  handle  daily  operations  of  the  IMRC,  but  also  to  provide 
training  to  MFL  faculty  (and  others  who  may  want  it)   in  the  use 
of  the  IMRC  facilities  and  in  the  development  of  instructional 
software.     On  a  part-time  basis,  a  number  of  students  assist  the 
director  by  handling  such  tasks  as  signing  out  headphones,  video 
tapes  and  discs,  and  programming  VCR's. 

The  Computing  Center,  LUCC,  is  still  actively  involved  with 
the  IMRC  by  providing  hardware  and  software  assistance  and  by 
providing  more  advanced  training  to  users  of  the  IMRC  computer 
classroom.     With  the  arrival  of  new  versions  of  software,  there 
is  an  ongoing  process  of  determining  that  all  hardware  and 
software  continues  to  interact  successfully.     It  is  still  a  goal 
of  this  project  to  place  all  software  on  a  write-protected  LAN  to 
ensure  the  integrity  of  the  software* 


Software  Development 

In  order  to  make  the  lab  functional  as  quickly  as  possible, 
LUCC  developed  a  number  of  programs  and  templates  which  can  be 
quickly  modified  for  MFL  use.     Since  the  MFL  department  already 
had  a  large  number  of  video  tapes,  the  first  LUCC-written  program 
simply  allowed  the  selection  of  which  video  source  would  appear 
on  the  computer  monitor;  VCR,   laserdisc,  or  live  broadcast. 
Without  this  program,  even  though  up  to  three  video  signals  may 
be  arriving  at  each  computer,  only  the  text  and  graphics  from  the 
computer  would  appear  on  the  computer  monitor. 

To  facilitate  courseware  development,  LUCC  created  a  number 
of  templates.     The  first  two  of  these  were  for  controlling 
laserdiscs;  one  for  CAV  discs  and  one  for  CLV  discs.     By  simply 
entering  titles  in  a  field  (column)  and  then  editing  laserdisc 
start  and  stop  locations  in  a  script  associated  with  that  field, 
a  series  of  video  clips  can  be  created.     Text  pertaining  to  all 
of  the  video  clips  can  be  placed  into  a  second  field  which  will 
automatically  scroll  to  the  proper  location  when  a  video  clip  is 
selected.     With  these  templates,  MFL  faculty  members  can  create 
courseware  for  any  foreign-language  laserdisc  with  relative  ease. 

One  of  the  primary  functions  of  the  old  MFL  language  lab  was 
the  ability  to  continually  listen  to  phrases  and  then  repeat  them 
into  a  microphone.     To  provide  this  functionality,  and  more, 
another  template  was  created  in  which  the  student  can  see  the 
phrase,  hear  the  phrase  and  repeat  it  into  a  microphone,  and  then 
hear  a  comparison  of  the  original  phrase  to  the  student's  own 
recording  of  the  phrase.     All  that  the  faculty  member  needs  to  do 
is  to  record  the  phrases  the  students  are  to  hear,  enter  the  text 
into  a  field,  and  then  edit  r  script  associated  with  the  text 
field  to  enter  the  name  of  the  files  containing  the  recordings. 


273 

ERIC 


8 


A  number  of  templates  were  created  for  questions  and  answers 
utilizing  multiple  choice,  f ill-in-the-blank,  and  matching 
formats.     Scoring  is  built  into  all  of  the  formats.     When  text  is 
entered  into  a  field,  an  attempt  is  made  to  check  for  minor 
spelling  errors  so  that  answers  won't  be  rejected  totally. 


The  Future  of  the  IMRC 

The  role  of  LUCC  with  the  IMRC  is  increasing  as  the  IMRC, 
along  with  the  rest  of  the  campus,  moves  from  a  combined  analog 
and  digital  environment  to  a  strictly  digital  environment.  The 
MFL  faculty  have  expressed  a  strong  interest  in  integrating  their 
own  personal  video  tapes  into  the  courseware  they  develop*  This 
can  probably  best  be  facilitated  within  a  digital  environment. 
The  testing  of  digital  devices  is  already  under  way. 


conclusion 

While  establishing  the  International  Multimedia  Resource 
Center  was  not  a  particularly  smooth  process,   it  did  prove  to  be 
a  valuable  learning  experience.     One  of  the  most  valuable  lessons 
was  to  buy  a  single  item  or  system  for  testing  before  buying  in 
bulk,  especially  if  the  technology  is  relatively  new  or  unknown. 

Through  the  establishment  of  the  IMRC,   issues  such  as  analog 
versus  digital  video,  and  what  is  the  minimum  acceptable 
resolution  and  number  of  colors  for  still  images,  are  now  being 
discussed.     While  superficial  discussions  regarding  campus-wide 
video  had  been  taking  place  for  years,   the  development  of  the 
IMRC  added  an  impetus  to  this  planning  and  forced  the  University 
to  more  closely  examine  the  integration  of  data,   audio,  and  video 
on  a  campus-wide  basis. 
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Abstract 

This  is  a  summary  of  an  introductory  programming  course  taken  by  pre-serviee  teachers  from  a  variety  of  fields.  Course 
development,  content,  assessment,  successes  and  failures,  and  student  reactions  are  described.  The  first  few  weeks  consists 
of  an  exposure  to  different  programming  paradigms  (Imperative,  Declarative,  and  Object-Oriented)  and  languages  and  their 
environments  (Turing,  Logo,  Prolog,  and  Smalltalk),  and  programming  principles.  The  remainder  of  the  course  consists  of 
a  student  project,  in  which  a  computer  system  is  specified,  programmed,  tested,  and  presented  to  the  class. 
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Development 
Clientele 

The  course  was  first  developed  by  the  author  (aLso  tiie  instructor)  in  the  winter  term  (January  to  May)  of  1991.  Two  groups 
of  pre-scrvice  teachers,  about  twenty  in  total,  have  followed  the  programme.  The  students  are  Bachelor  of  Education  and 
Diploma  in  Technical  Education  candidates  at  the  Faculty  of  Education  at  Queen's  University  who  attend  the  institution 
from  Septembei*  to  May.  Admi.ssion  to  the  Faculty  is  ba.sed  on  equal  weighting  of  both  academic  and  personal  qualities  of 
the  applicant,  and  competition  is  intense  to  gain  entrance.  All  students  require  a  course  in  psychology,  and  there  are 
minimum  numbers  of  subject-specific  courses  needed  for  admission  into  B.Ed,  subject  sixjcializations  at  the  intermediate 
and  secondary  levels.  Queen's  can  typically  admit  less  than  one  sixth  of  all  those  applying,  and  in  some  specialities  the 
ratio  is  even  greater.  These  students  possess  great  diversity  in  both  academic  and  personal  experiences^  The  mean  age  of 
students  at  the  Faculty  is  over  28  yeiirs. 

Students  liike  several  different  courses,  depending  on  their  preferred  teaching  level  (e.g.,  primary-junior,  junior- 
intermediate,  or  intermediate-secondary).  Some  of  iliese  courses  focus  on  curriculum  in  their  chosen  specialities,  some  deal 
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with  foundations  issues  such  as  psychology,  philosoj)hy,  elc.»  others  deal  with  professional  skills  including  curriculum 
design,  assessment,  computer  and  media  skills,  and  so  on»  and  finally  some  courses  arc  offered  purely  as  elcciives.  A  total 
of  nine  weeks  of  practice  leaching  rounds  is  distributed  over  the  year,  typicaJly  in  two-week  stays  at  a  school.  During  these 
limes  all  courses  at  the  Faculty  are  interrupted  while  student  teachers  and  Faculty  instructors  work  in  the  schools.  Elective 
courses  last  one  term  in  length,  thirteen  weeks  of  class  time  are  broken  up  by  practice  teaching.  The  course  described  here 
is  an  elective. 

Rationale  and  Genera)  Aims 

No  assumptions  could  be  made  about  either  student  background  knowledge,  or  their  aims  in  taking  this  course.  Student 
participants  have  come  from  such  diverse  areas  as  elcmcntiu-y  education,  music,  physical  education,  mathematics,  science* 
and  computer  science.  Some  students  took  the  progianwning  course  to  prepare  them  for  later  professional  qualifying 
courses  for  the  leaching  of  computer  studies,  some  because  they  knew  something  already  about  programming  and  wanted  to 
deepen  their  understanding  for  personal  satisfaction,  and  others  decided  that  this  was  a  good  opportunity  for  Ihcm  to  start  to 
learn  about  programming. 

The  course  was  developed  to  give  the  students  an  overview  of  what  programming  a  computer  is  about,  but  in  a  way  that  was 
quite  Oiifcrenl  from  many  other  programming  courses.  Most  traditional  courses  focus  on  a  single  programming  language 
and  paradigm  and  treat  them  in  some  depth,  to  give  students  a  knowledge  and  capability  with  that  particular  system.  The 
syntax  and  structure  of  one  programming  system  are  experienced  in  detail.  Given  that  most  of  (hcse  students  will  not  go  to 
more  advanced  courses,  and  that  some  will  have  never  done  any  programming  before,  it  was  decided  to  provide  a  bird's-eye 
view  of  the  area,  rather  than  a  pedestrian's.  Granted,  taking  a  helicopter  ride  up  Mount  Everest  does  not  give  one  the  same 
cutting-edge  appreciation  for  the  technical  difficulties  laced  by  climbers,  but  it  docs  show  the  lay  of  the  land  in  ways  that 
cannot  be  effectively  perceived  by  tliose  following  tortuous  routes.  The  goal  of  this  course  was  a  forcsl-view,  not  a  trees- 
view.  This  was  deemed  to  be  more  attainable  and  beneficial  for  both  programmers  and  non-programmers. 

One  component  of  the  overview  was  a  look  at  the  expressive  power  of  programming  systems.  A  computer  program  is 
usually  crafted  to  solve  some  problem.  There  is  a  two-step  process  at  work.  First,  the  problem  in  the  real  world  is  isolated 
and  described  in  simpler  conceptual  terms.  A  transformation  from  the  complexity  of  the  real  world  to  a  distilled  essence  of 
the  related  ideas  is  sought.  It  is  in  this  form  that  the  problem  is  really  solved  in  an  algorithmic  sense.  Many  generic  tools 
can  be  brought  to  bciy  on  the  problem  at  this  stage:  features  of  the  model,  negligible  factors,  and  above  all,  expression  of 
the  conceptual  problem  in  mathematical  terms  and  subse(|ueni  manipulation  in  this  form.  Much  can  go  wrong  in  this 
process.  Assumptions  may  be  inaccurate  or  inadequate,  the  simplified  concepts  may  not  be  capable  of  representing  the 
problem  fairly  or  completely,  mathematical  operations  may  be  misused  or  abused. 
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very  complicated  highly  simplified       /    software  equivalent 

Figure  1:  Problem  Solving  by  Compulcr:  A  Two-Slcp  Process 

The  second  slcp  in  the  process  is  ihe  casting  of  the  conceptual  model  into  software  that  is  executed  by  hardwcU'e.  This  is  not 
always  a  trivial  task.  The  representation  in  software  must  be  valid  with  respect  to  the  conceptual  model.  Most  modern 
programming  systems  do  include  a  host  of  mathematical  operations  that  can  carry  out  those  required  by  a  conceptual 
solution  algorithm.  Unfortunately,  they  do  not  always  allow  the  freedom  of  expressiveness  needed  by  conceptual  models. 
How  does  the  programmer  capture  the  features  of  the  conceptual  data  in  a  programming  language?  In  most  systems  there  is 
a  still  quite  a  narrow  range  of  built-in  data  structures  that  can  hc-i  these  data,  but  conceptual  data  of  arbitrary  complexity 
often  has  to  be  jammed  into  bizarre  combinations  of  data  structures.  This  alone  can  lead  to  errors.  The  conceptual  data  can 
be  misrepresented  in  software.  Program  execution  may  mean  using  dati'  piecemeal,  which  can  lead  to  incorrect  exU'actions 
of  the  required  parts  of  the  data,  or  unwanted  side-effeciii  when  operations  are  performed.  Finally,  the  software 
rcpresentation  of  the  data  is  manipulated  by  operations  specified  by  the  flow  of  execution  of  a  sequence  of  instructions.  In 
many  systems  the  data  are  separated  from  the  kind  of  behaviour  it  should  have;  data  are  sent  to  procedures  for  some 
operation  to  be  performed,  and  tliey  are  returned  in  some  modified  form.  Representing  the  steps  in  a  problem  solution  is 
not  always  easily  embodied  in  a  programming  system. 

The  expressiveness  of  programming  systems  was  sampled  in  two  ways.  One  way  was  through  the  features  of  the  language 
itself,  especially  with  respect  to  the  kinds  of  data  that  could  be  represented.  Data  included  (in  a  general  sense)  numbers, 
text  strings,  graphics,  and  sound.  Secondly,  and  more  importantly,  several  programming  paradigms  were  examined.  A 
programming  paradigm  in  this  sense  refers  to  the  way  in  which  a  problem  is  redefined  in  software  and  the  mechanism  in 
which  the  created  software  arrives  at  a  solution  to  the  problem.  Four  languages  (Turing,  Logo,  Prolog,  and  Smalltalk)  and 
three  paradigms  (imperative,  declarative,  and  object-oriented)  were  used. 

Whatever  programming  system,  issues  of  program  specification,  design,  structure,  modularity,  and  testing  were  considered 
to  be  important.  Some  key  sofiwcire  engineering  ideas  were  introduced  as  the  course  progressed.  Mechanical 
considerations  of  programming  environment  were  also  significant  features,  A  comparison  of  screen  layout,  editor 
operation,  compiler  or  interpreter  invocation  and  commands,  debugging  tools  was  included  in  the  course. 

For  the  sake  of  completeness,  so  that  students  could  have  some  idea  of  the  relationship  between  their  programming  and  the 
execution  of  code  at  the  lowest  levels  in  the  machine,  basic  ideas  of  the  various  components  of  a  programming  were  briefly 
described.  This  involved  no  more  than  a  summary  of  the  relationship  between  editors,  compilers/interpreters,  machine- 
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level  code,  and  operating  systems.  This  was  only  of  j)assing  interest  to  complete  novices,  and  to  more  experienced 
programmers,  but  added  some  context  to  intermediate  programmers  who  could  now  appreciate  some  of  this  terminology. 
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Figure  2:  Componcnis  of  a  Programming  System 

Finally,  it  was  critical  that  students  appreciate  the  creative  side  of  programming.  Programmers  do  analyze  problems.  But 
the  extraordinary  intellectual  pleasure  of  programming  comes  from  the  synthesis  of  a  solution  design,  its  implementation, 
and  its  execution.  To  experience  the  piciisure  of  seeing  one's  own  creation  drive  a  machine,  which  can  be  a  tremendous 
mental  lever,  is  to  feci  the  power  and  joy  of  the  human  side  of  programming.  Computing  can  be  an  awful  lot  of  fun! 

Key  Elements:  The  Languages  and  Paradigms 

The  languages  include  Turing  (a  Pastal-hkc  structured  language  that  includes  coordinate-style  pixel  graphics  and  sound) 
[Holt  92],  Logo  (a  language  that  includes  both  Turtle  graphics,  sound,  and  Lisp-like  dal^  structure  processing)  [Papcrt  80]. 
Both  these  languages  use  the  imperative  paradigm,  typilicd  by  Niklaus  Wirth's  dictum  algorithm  +  data  structures  = 
programs  [Wirth  76].  That  is,  the  programmer  must  arrange  a  controlled  flow  of  data  through  a  prepared  sequence  of 
operations  to  have  the  program  produce  the  intended  result.  It  falls  to  iho  programmer  to  create  the  software  that  structures 
and  contains  the  data  for  the  ideas  dealt  with  (usually  something  in  the  real  world),  and  to  know  what  operations  to  perform 
on  these  data  and  how  to  do  achieve  them.  These  are  not  trivial  tasks,  even  when  the  desired  outcome  of  program  execution 
is  clearly  understood.  Turing  is  very  much  a  "conventioiial"  programming  language  in  its  form,  but  has  a  remarkably  clean 
syntax  (something  like  scmi-colon-less  Pascal).  It  has  the  usual  data  types,  integers,  reals  (floating  point),  strings  of 
characters,  arrays,  and  records.  Variables  must  be  declared  and  as  such  there  is  strong  type  checking.  Variables  can  have 
both  global  (system-wide)  and  local  (within  a  procedure)  scope,  a  common  feature  of  this  generation  of  languages.  It  can  do 
pixel  graphics  in  multicolours  based  on  standard  coordinate  geometry  with  an  origin  in  the  lower  left-hand  comer  of  the 
screen.  Music  is  easy  to  add  by  a  frequency/duration  specification.  There  is  a  small  vocabulary  (ca.  thirty)  of  reserved 
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words  for  Turing  commands.  Turing  programs  arc  wriiicn  by  using  these  primiiivcs  within  a  sequence  of  short  one  line 
commands,  and  by  grouping  sections  of  Uiese  into  named  and  callable  procedures.  Each  of  these  programming  systems  use 
a  different  user  interface  for  execution  and  development.  While  both  incorporate  editors  that  are  integrated  with  the  system, 
Turing  provides  synutx  checking  before  run  time  (Turing  code  can  be  compiled)  that  will  not  allow  escape  from  the  editor 
until  such  errors  are  corrected.  Programs  are  created  and  modified,  then  run,  by  a  separate  command  from  within  the 
editor.  The  programmer  does  not  need  to  know  that  the  code  is  checked,  and  that  library  routines  may  be  linked,  all  by  the 
"run"  command  from  within  the  editor.  Af;er  execution  the  system  returns  to  the  editor.  Edited  code  can  be  saved  or 
retrieved  in  files  froivi  within  the  editor. 

Logo  is  best  known  for  its  "Turtle  graphics"  in  which  commands  are  given  to  a  conceptual  Turtle  object  that  drags  a 
multicolourable  pen  across  the  screen  (an  abstraction  ol"  a  real  robot  Tunic  that  moves  along  the  fioor  carrying  a  retractable 
pen).  Gcomeu-y  is  Turtle-centred  and  di!Terential  (for  the  most  part).  That  is,  command  sequences  used  to  create  figures 
are  centred  on  the  Turtle  itself,  its  final  position  depending  on  its  initial  position  and  orientation  and  the  self-referencing 
commands  to  turn  Uirough  an  angle  or  take  several  Turtle  steps,  as  compared  with  specifying  a  move  to  a  Cartesian  point  on 
the  screen.  The  basic  data  structure  is  the  list,  and  Logo  is  very  Lisp-like  in  how  symbolic  or  mathematical  procedures  are 
implemented.  Variables  do  not  have  to  be  declared  in  Logo,  and  strong  typing  is  not  required.  Furthermore,  variables  can 
vOnly  have  global  scope  (they  cannot  be  declared  local  to  a  procedure).  Some  limiuitions  of  this  feature  can  be  overcome  by 
using  arguments  when  calling  procedures.  We  use  a  version  of  Logo  called  LogoWriter  [LCSI  86J.  The  LogoWriter  editor 
is  more  rudimentary  than  that  of  Turing,  but  is  more  seamlessly  connected  to  the  system.  Using  the  editor  involves  going  to 
the  "flip  side"  of  the  "page",  in  which  all  new  code  is  written.  The  user  must  return  to  the  "front  side"  of  the  LogoWriter 
"page"  to  call  the  code  for  execution.  Logo  code  is  not  checked  extensively  on  exit  from  the  editor.  LogoWriter  pages  can 
be  saved  in  files  for  later  recall  or  editing  in  another  LogoWriter  session.  Logo  also  possesses  an  immediate-mode 
execution  facility  from  which  users  not  only  invoke  program  execution  but  write  and  execute  short  "one-liners"  for 
investigative  purposes.  There  is  again  a  small  vocabuhiry  (ca.  thirty)  of  reserved  words  for  Logo  commands;  Logo 
programming  consists  of  extending  the  language  by  creating  new  "words"  that  invoke  combinations  of  the  primitive 
operations  or  other  previously-defined  procedures. 

Since  both  languages  rely  on  the  How  of  execution  conu-ol  they  can  illustrate  features  of  structured  programming  and 
modularity.  Structured  programming  is  an  approach  to  restricting  How  of  control  to  three  main  types:  the  sequence,  the 
conditional  lest  (if ...  then  ...  else)  and  the  loop  (do  ...  while  or  repeat  ...  until  Structured  programs  are  easier  to 
understand  and  maintain  than  those  that  are  not,  particularly  those  that  contain  "goto"  statements  which  can  redirect  flow 
arbitrarily.  Modular  programming  encourages  the  packaging  of  code  segments  into  a  group  of  code  devoted  to  a  particular 
operation.  When  the  operation  is  required  for  execution  the  control  passes  to  the  module  and  its  contained  code,  and  then 
returned  to  the  part  of  the  program  that  called  it. 

The  other  programming  systems  usal  in  the  course  are  Prolog  uiul  Smalltalk.  Prolog  embodies  the  declarative  paradigm 
espoused  by  Kowalski:  al^oriihm  =  lo^ic  +  conirol  |Kowalski  79|.  The  name  of  the  language  comes  from  p/-(?gramming 
with  logxz.  In  this  context,  the  role  of  the  programmer  is  to  s|)ccify  the  relationships  between  data  that  must  satisfy  the 
solution,  then  let  the  Prolog  inference  mecnanism  (built  in  to  the  Prolog  system)  direct  control  until  the  data  are  massaged 
such  that  the  solution  is  found.  That  is,  the  progi'aminer  need  not  know  how  to  reach  a  solution  in  a  procedural  way,  only 
tlie  constraints  that  must  be  satisfied  by  a  solution.  Of  course,  it  isn't  quite  thai  simple.  The  programmer  must  understand 
how  the  underlying  control  mechanism  works  to  arrange  die  facts  and  rules  that  comprise  the  program  so  that  they  will 
generate  the  necessary  daui  modifications.  The  algorithm,  or  sequence  of  operations,  is  a  result  of  the  program  (database) 
of  logical  statements  combined  with  the  built-in  conu*ol  mechanism.  Prolog  is  not  a  system  that  is  well-suited  to  number 
crunching,  and  since  it  is  not  structured  inherently  for  control  it  does  not  lend  itself  to  structured  programming 
methodologies.  However,  the  database  of  i'aeis  and  rules  can  be  modularized  to  encourage  low  coupling  of  components  of 
the  overall  logic,  which  enhances  readability  and  reduces  maintenance  problems. 
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The  paradox  that  ihc  code  is  specified  dcclaraiively  and  yei  executed  procedurally  is  ihe  characierisiic  of  the  system  that 
confuses  most  beginning  Prolog  programmers.  Prolog  has  a  small  set  of  primitives.  Prolog  programs  are  databases  of  facts 
and  rules  that  augment  and  call  one  another  and  ihe  basic  system  primitives.  The  programmer  cannot  simply  "read"  Prolog 
code  sequentially,  as  in  an  imperative  language,  because  execution  order  is  based  on  the  inferencing  mechanism  that  directs 
the  flow  of  control  ai  run  lime.  The  best  that  the  programmer  can  do  is  lo  create  the  necessary,  self-contained  facts  and 
rules  that  target  a  specific  outcome  (usually  two  or  three  short  lines).  It  is  the  entire  database  of  these  clauses  that  makes  up 
the  elements  of  a  working  "program".  There  is  no  integrated  editor  with  most  versions  of  Prolog,  though  the  system  can  be 
set  up  to  invoke  an  external  editor  from  an  internal  call  to  the  immediaie-niode  interpreter.  The  editor  is  simply  a  text 
editor  operating  on  an  external  file;  when  it  starts  up  Prolog  relinquishes  all  control  to  it  until  it  is  exiled.  On  exiting  the 
editor  thr^  programmer  returns  immediately  to  the  Prolog  system.  To  some  extent,  liiese  shifts  are  transparent,  but  they  are 
awkwarK-*  U  is  also  possible  to  edit  Prolog  files  completely  independendy  of  the  system  and  later  file  in  the  database  without 
recourse  to  an  editor  call.  To  add  to  or  modify  the  existing  database  of  facts  and  rules  Prolog  must  be  given  a  goal  to 
consult  the  previously  edited  file.  On  reading  the  lilo  l^iolog  will  declare  certain  syntax  errors  that  it  cannot  parse. 

In  the  version  of  Prolog  that  we  use,  Wisdom  Prolog  IMl'I'  J^rcss  1991 1,  there  are  three  screen  windows.  The  windowing 
environment  is  entirely  text-orienteil,  not  graphics,  and  is  written  in  Prolog.  One  is  a  general  "console"  that  permits  the 
user  to  solve  problems  aail  give  directives  directly  to  the  Prolog  interpreter.  It  is  used  to  consult  files  of  facts  and  rules  (to 
bring  them  into  tlie  system  database  of  all  facts  and  rules)  and  to  give  the  interpreter  goals  that  it  must  solve.  Another 
window  can  be  used  to  create  "one  off  databases  of  facts  or  rules  for  testing  purposes,  though  it  is  not  recommended  as 
these  data  cannot  be  subsequently  saved  to  a  file.  Fituilly,  tliere  is  a  trace  window  that  will  track  each  fact  and  rule  call 
made  by  the  inference  engine  during  execution.  This  can  be  useful  for  debugging. 

Smalltalk  is  an  object-oriented  language  in  its  purest  form  [Goldberg  84).  In  the  object-oriented  paradigm  problems  are 
solved  by  creating  software  structures  (objects)  that  contain  the  state  (data)  and  behaviour  (procedural  code)  of  the 
components  and  then  allow  them  to  interact  through  message  sending  and  responses  among  themselves.  In  a  sense,  object- 
oriented  programming  is  problem  solving  by  simulaiiofu  Object-oriented  systems  also  make  use  of  the  idea  of  classes  of 
objects  which  are  templates  from  which  specific  instances  objects  can  be  created,  but  which  contain  the  executable  code  for 
them.  Moreover,  classes  are  arranged  in  a  hierarchy  so  that  related  classes  of  greater  specificity  (subclasses)  can  inherit 
state  and  behaviour  from  those  of  greater  generality  (suj)erclasses).  Smalltidk  is  a  big  system.  In  the  version  that  we  use, 
Smalltalk  286,  there  are  over  one  hundred  classes  and  about  three  thousand  "methods"  (procedures)  associated  with  the 
basic  system  [Digitalk  88|.  Programming  in  Smalltalk  is  done  by  extending  and  specializing  existing  classes  or  creating 
new  ones,  and  by  reusing  as  much  existing  system  code  as  possible. 

A  typical  "program"  execution  is  started  by  asking  Smalltalk  to  create  and  iniiiali/.e  an  instance  of  some  class,  and  sending 
it  a  start-up  message.  Execution  proceeds  by  a  subsci[uent  barrage  of  message  sends  (usually  many  hundreds)  to  various 
objects  in  the  system  such  as  text  windows,  etc.  Much  of  the  low-level  complexity  is  hidden  from  the  programmer,  who 
only  has  to  know  the  "externar"  protocol  that  an  object  responds  to.  With  so  much  reuse  of  code,  and  the  organization  based 
on  a  class  hierarchy,  "programs"  in  Smalltalk  are  usually  distributed  throughout  the  system.  Thus  the  programmer  cannot 
"read"  Smalltalk  code  very  easily,  as  it  is  broken  into  separate,  very  small,  behavioural  chunks  for  the  objects  for  which  they 
are  written.  Rather,  the  programmer  browses  and  augments  the  various  components  of  behaviour  in  small  chunks. 
Smalltalk  permits  the  building  of  a  class  of  object  of  arbitrary  complexity,  so  there  is  no  resuiction  at  all  on  data  types, 
though  many  variations  are  built  in  (integers,  reals,  collections  -  including  bags,  sets,  arrays,  ordered  lists,  and  swings). 
Smalltalk  is  not  suongly  typed,  as  variable  binding  occurs  at  run  time,  when  the  message  is  sent.  Global  and  local  variables 
are  permitted. 

The  Smalltalk  environment  is  built  entirely  in  bit-mapped  graphics.  All  "programming"  is  done  by  interacting  with 
graphics  windows  and  the  mouse,  cutting,  pasting,  augmenting  and  browsing  system  code.  Nearly  all  the  Smalltalk  system 
is  written  in  Smalltalk,  and  the  programmer  can  sec  and  modify  any  of  it;  there  is  no  operating  system,  as  such,  with 
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Smalltalk.  There  is  a  window-based  dynamic  debugging  facility  built  in  lo  Smalltalk  that  permits  inspection  and 
modification  of  objects  during  execution. 

Data  and  Procedure  Roprcscnlution  "Fit"  with  Problem  Domain 


Pro}}ramminj4  Knvironmciit 


Figure  3:  Programming  Paradigms  and  Langua^jcs 

Just  as  there  are  many  diflercni  kinds  ol'  problems  that  can  be  approached  through  computer  programming,  so  goes  the 
difficulty  of  finding  a  single  language  or  paradigm  that  will  allow  the  programmer  to  solve  all  problems  with  equal  facility. 
Some  languages  have  numerical,  string  handling,  graphics  representation,  or  sound  generation  features  that  make  ihem 
more  s*iiti\ble  for  some  problems  as  compared  with  others.  That  is  not  to  say  that  one  could  not  attack  any  problem  in 
virtually  «iny  programming  system.  It  is  a  question  oC  tliecasc  with  which  one  can  map  features  in  the  problem  domain  into 
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software.  The  expressiveness  of  a  programming  system  depends  on  more  than  just  language  syntax  and  structure,  though 
that  is  part  of  the  difference.  Turing  and  Logo  can  both  do  superb  graphics,  but  Logo  Turtle  graphics  is  more  suited  lo 
differential  geometry  than  Cartesian-based  pixel  graphics.  A  siiigular  difference  among  systems  can  be  seen  in  the 
paradigm  of  computation:  how  data  are  represented  aiul  Interact.  Prolog  handles  expert  system  construction  and  certain 
constraint  problems  much  more  easily  than  an  imperative  language,  simply  because  of  its  separation  of  the  knowledge  and 
facts  from  the  engine  that  drives  the  reasoning.  Smalltalk  offers  uninhibited  design  of  any  data  structure  coupled  with 
operations  on  that  data  structure  (encapsulated,  so  that  the  interna!  implementation  can  be  hidden  from  other  objects). 
Smalltalk  objects  are  "active  data",  which  respond  to  messages*  sent  them  depending  on  their  programmed  behaviour.  It 
allows  the  programmer  to  concentrate  on  capturing  the  essence  of  the  world  in  software  objects  that  relate  intimately  and 
directly  with  their  real-world  counterparts.  There  is  no  need  to  divide  complex  things  and  jam  them  into  a  few  data 
structures  offered  by  the  programming  system.  Smalltalk  is  especially  suited  to  simulation  and  modelling  of  large  systems 
of  complex  objects. 

Programming  in  parallel  systems  anJ  conncciionisi  computing,  while  interesting  and  different  paradigms  of  great 
significance  in  their  own  righis,  are  deemed  to  be  beyond  the  scope  of  the  course. 

The  general  aims  of  the  course  are  summari/od  by  the  following  general  objectives,  as  given  to  the  students. 
Tlie  student  will  be  able  to; 

1.  describe  the  (i)  process  of  programming 

(ii)  major  programming  paradigms  (and  an  example  of  each) 

(iii)  importance  of  programming  environment  features 

(iv)  fundamenUil  concepts  of  good  progratn  design 

2.  design,  implement,  debug,  and  present  a  programmmg  project  in  the  student's  language/system  of  choice  (in  agreement 
with  the  instructor) 

Topics 

The  first  half  of  the  course  consists  of  an  exposure  to  different  programming  paradigms  (three)  and  languages  and  their 
environments  (four),  and  program  design  principles.  The  remaining  part  of  the  course  consists  of  a  student  project,  in 
which  a  problem  is  specified,  programmed,  tested,  and  presented  to  the  class.  The  project  is  chosen  in  mutual  agreement 
with  the  instructor.  Times  shown  are  for  in-class  sessions,  at  the  keyboard,  of  which  three  hours  are  scheduled  each  week. 
Often,  out-of-class  time  must  be  added  as  well. 

Week  1:  Introductions.  Overview  of'  programming  paradigms,  languages,  and  environments  (1  h  total).  The  Turing 
language  (5  h  total):  environment,  variables  and  types,  input  and  output,  processing,  assignment,  if/then/e'sc  structures. 

Week  2:  Touring  Turing:  loops  (counted  and  conditional  exit),  random  number  generation,  character  strings,  modules 
(procedures  and  functions),  graphics  and  sound.  Assignment  #1 :  Turing. 

Week  3:  The  LogoWriter  language  (4  h  total):  introduction,  environment,  turtle  graphics,  repeat  loops,  state  transparency, 
modules  (procedures,  procedures  with  inputs),  top-down  and  bottom-up  design,  if/then/else  structures. 

Week  4:  The  last  of  Lc;|^oWntcr  :  recursion,  numerical  procedures,  list  processing  (including  strings  and  sentences), 
music.  Assignment  ii2:  I.ogoWriier.  The  Prolog  language  (4  h  total):  introduction,  environment,  database  of  facts  and 
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rules,  closed-world  assumptions,  inference  engine,  unification  and  instantiation,  numerical  calculations,  recursion,  search, 
sound. 

Week  5:  Completing  Prolog:  generate  and  test,  list  jhocessing,  expert  systems.  Assignment  #3:  Prolog.  The  Smalltalk 
language  (4  h  total):  introduction,  environment,  objects,  messages,  classes,  encapsulation,  inheritance,  programming  by 
simulation,  code  fragments. 

Week  6:  Saying  goodbye  to  Smallliilk:  data  nbsiraciion,  class  libraries,  magnitude^  collection,  and  graphics  classes,  model- 
view-controUer  paradigm,  creating  a  small  counter  application,  investigating  and  modifying  a  complex  simulation. 
Assignment  M. 

Weeks  7  -  Program  design  issues.  Discussion,  choosing  of  projects,  work  begins.  Project  work  is  mostly  independent. 
We  save  the  last  couple  of  classes  for  student  presentations  of  projects. 

The  course  was  thirteen  weeks  long. 

The  aim  of  the  project  portion  of  the  course  was  to  give  practical  experience  in  designing  and  implementing  a  program  to 
solve  a  problem  that  was  of  interest  to  the  student.  They  chose  any  programming  language  and  any  computer  for  their 
project.  Some  effort  was  devoted  to  learning  a  new  system,  or  gaining  other  information  that  enabled  the  solution  of  the 
problem.  The  students  verified  that  their  projects  were  satisfactory  in  terms  of  degree  of  difficulty;  i.e.,  limited  enough  in 
scope  and  complexity  that  tliey  could  be  done  in  a  reasonable  lengtli  of  time,  yet  involved  enough  to  stretch  their  abilities. 

Previous  student  projects  have  included:  a  spreadsheet  application  with  macros  in  u*acking  basketball  statistics,  HyperCard 
(Apple  Macintosh  system)  stacks  in  social  activities  (P.H.E.)  and  biology,  Toolbook  (a  hypermedia  program  for  the  IBM  PC 
family)  programs  in  history,  health,  biology,  chemistry,  physiology,  face  reconstruction,  a  Turing  version  of  the  game  of 
Life,  and  a  hyperspace  graphics  generator,  a  Logo  graphics/music  player,  a  Prolog  fuzzy-logic  investment  counselor,  a 
garden  planner,  and  an  automobile  diagnosis  expert  system  (partial),  and  a  Smalltalk  network  planner  and  complex  number 
class. 


There  were  no  lectures.  Aside  from  a  brief  introduction  to  the  general  properties  of  the  programming  system  being 
sampled,  there  were  no  formal  class  discussions.  Students  worked  almost  exclusively  at  the  keyboard,  in  pairs  or  alone.  A 
comprehensive  set  of  notes  guided  them  through  examples,  concepts,  problems,  and  explorations.  Reference  manuals  were 
kept  in  the  computer  lab  at  the  Faculty  of  Education;  sometimes  liiey  were  signed  out  on  an  individual  basis,  quantilies 
permitting  (we  had  more  students  than  computers  and  programming  systems,  and  we  remained  within  licensing  limits). 
Students  were  usually  preseni  during  scheduled  class  periods,  but  not  always,  .  Attendance  was  not  a  requirement  except 
for  project  presentations.  Some  students  worked  at  home  on  their  own  computers,  if  they  had  them;  they  purchased  their 
own  copies  of  Turing  at  a  very  low  price,  and  our  site  licence  for  LogoWriter  allowed  them  to  use  the  system  at  home  while 
they  were  students  at  the  Faculty  of  Education.  A  couple  of  students  also  had  their  own  copies  of  Prolog  (^Turbo  Prolog). 
No  student  has  ever  had  Smalltalk  on  their  own  computer.  The  insu*uctor  was  available  at  each  scheduled  class  meeting, 
and  often  outside  those  hours,  for  assistance.  This  was  seldom  needed,  as  there  was  usually  peer  consulting  going  on  within 
the  group  about  how  to  get  past  particular  problems.  The  instructor's  main  activity  was  to  unobtrusively  join  the 
programmers  and  eavesdrop,  commenting  on  the  thinking  that  was  going  on,  or  suggesting  alternate  ways  of  approaching 
the  difficulty.  The  instructor  was  not  the  problem  solver  or  the  resident  expert  who  would  dispense  shortcut  solutions  and 
clever  hacks.  The  instructor  was  a  mediator  in  analysis,  design,  and  debugging. 
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Besides  ihcse  meetings,  there  was  little  lormal  interaction  between  instructor  and  student.  At  the  scheduled  end  of  the  time 
allocated  lo  each  programming  system  insuuctor  and  siudcnts  would  mutually  agree  on  a  due  date  for  the  assignment. 
Assignments  would  be  handed  in,  theti  commented  upon  and  marked  by  the  instructor,  and  returned  promptly  by  the  next 
class  meeting. 

There  was  a  class  meeting  to  discuss  the  project,  its  assessment,  and  show  examples  of  previous  projects.  After  that  point 
students  worked  whenever  they  wished  to  and  consulted  ihc  instructor  on  an  ad  hoc  basis.  An  extended  class  was  scheduled 
for  the  last  meeting,  at  which  students  presented  their  programs  to  the  group.  This  celebration  was  the  culmination  of  the 
course. 

Assessment 

All  assignments  and  projecus  were  handed  in  as  boili  hard-copy  and  machine-readable  form  (on  disk).  The  assessment 


Four  given  problem/programming  assignments 

(one  on  eacli  language:  Turin?^,  Lo^oWriter,  Prolog,  and  Smalltalk) 

40% 

Programinmg  project: 
Design,  program,  documentation  =  40% 
Class  presentation  =  20%  (partly  student  assessed) 

60% 

Total  Mark 

100% 

scheme  was  given  out  with  each  assignment  or  project. 


Assignments  were  handed  m  on  mutually  agreeable  due  dates.  To  allow  quick  turnaround  of  assignments,  to  avoid  needless 
procrastination,  and  to  be  I'air  to  tliosc  who  complete  work  on  time,  late  assignments  had  the  total  possible  score  reduced  by 
20%  per  late  day. 

We  used  the  following  gracing  system,  based  on  student  preference:  Pass  (No  Honours)/Fail.  The  breakdown  would  be 
based  on: 


Pass 

5iWi  <  Total  Mark 

Fail 

Touil  Mark  <  50% 

(Occasioi.nlly  students  will  choose  an  Honours/Pass/Fail  system.) 

A  typical  assignment  in  Turing  and  LogoWriter  consisted  of  designing,  implementing,  and  testing  four  different  programs. 
Of  the  four,  one  was  usually  based  on  numerical  calculation,  one  on  some  control  feature  of  the  language,  one  of  some  form 
of  data  type  or  structure,  and  one  on  graphics  or  sound  (usually  there  is  a  choice).  The  Prolog  assignment  usually  consisted 
of  databases  already  given  the  student  for  modification  ur  augmentation  to  another  purpose.  For  example,  a  generate  and 
test  program  was  modified,  and  a  tiny  cx|)c.t  classifier  system  was  created  (facts)  using  a  shell  given.  The  Smalltalk 
assignment  consisted  ofbroNvsing  the  system,  finding  and  explaining  code,  and  extending  some  existing  application  classes. 
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Each  assignment  was  assessed  according  lo  ihe  following  criieria: 
All  programs  should  coniain  docuinenuuion  thai  includes: 

•  Name  of  progranimer 

•  Title  of  program,  and  the  date 

•  A  general  comment  on  what  the  program  docs  and  how  it  does  it 

•  A  list  of  variable  names  and  what  each  variable  represents 

•  Comments  sprinkled  u^irough  the  program. 
Program  s  are  cva  1  uated  based  on : 


•  Program  Dcai^n 

clarity  10 

correctness  (meets  specifications)  5 

efficiency  5 

•  Program  Documeniaiion 

name,  title,  date,  general  statement  5 

meaningful  variable  names  and  description  10 

general  com  menus  5 

•  Program  Exccuiion 

correctness  (mccLs  specifications  and  runs)  10 

•  Total  50 


The  aim  of  the  project  portion  of  the  course  was  to  give  practical  experience  in  designing  and  implementing  a  program. 
The  project  deliverables  included  a  description  of  the  problem,  a  summary  of  the  approach  that  was  used  to  solve  it,  and 
what  tools  (programming  languages  or  hardware)  and  algorithnjs  were  brought  to  bear  on  the  problem  (and  why).  They 
also  included  the  program  or  system  code.  Finally,  the  students  presented  their  problem  and  its  demonstrated  computer 
solution  to  the  class  in  a  time-limited  (15  minute)  session  ai  the  end  of  'he  course. 

The  marking  scheme  for  the  project  is  shown  below.  Students  arc  involved  in  assessing  their  colleagues'  work  for  part  of 
the  total  score. 

Overall: 

Creativity  and  originality:  Is  it  a  good  problem,  suitable  to  solution  by  computer,  and  can  you  sec  why 
someone  would  find  it  interesting?  10% 

Degree  of  difficulty:  Did  the  problem  stretch  the  author's  (not  your)  problem  ::olving  and  intellectual 
capabilities?  (The  solution  may  fall  lar  short  of  perfection,  but  it  may  have  been  very  challenging  to  the 
author)  10% 
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Effort:  Did  ihc  problem  rcprcsciu  subsUiniial  ciTori  on  the  auilior?  (Ii  may  have  been  a  fairly  simple 
inielleciuol  problem,  but  a  tremendous  effort  went  into  an  outsumding  product.)  5% 

Success:  Was  the  problem  solved  completely  and  effectively  by  ihe  program?  10% 

Documentation  and  Pro  ^ramming : 

Problem  statement:  Is  it  clear,  and  unambiguous,  and  does  it  define  the  scope  of  the  work?  5% 

Description  of  approach:  Were  ihv  choices  oC  plaiform  and  programming  system  /  language  reasonable, 
and  were  die  algorithms  suitable?  10% 

Program  clarity,  design  I  structure,  appearance,  commenting:  Is  the  program  understandable?  Docs  it 
break  down  complex  problems  into  smaller  ones?  Are  meaningful  variable  names  usctl?  Is  llie  program 
readable?  20% 

Program  correctness:  Df^cs  ihe  program  do  whai  it  is  supposed  to  do?  Does  it  uap  common  errors?  10% 

Presentation  (15  minutes): 

Clarity:  Is  the  problem  clearly  stated?  5% 

Completeness:  Is  the  presentation  complcie,  in  ihat  you  can  see  how  the  demonstration  of  the  computer 
program  solves  the  problem?  5% 

Motivation  level:  Is  the  presejuation  motivating;  would  you  like  to  see  and  hear  more  about  this  problem 
and  program?  5% 

Timing:  Is  the  presentaiion  well-timed;  does  the  presenter  move  effectively  from  one  topic  to  the  next 
within  the  overall  time  limit?  5% 


Different  computers  have  been  used  in  the  course,  including  Unisys  ICON  II,  Apple  Macintosh,  and  Zenith  IBM  PC  386 
compatibles.  The  platform  of  choice  has  settled  on  die  PC,  mainly  because  Uiere  is  a  small  lab  of  eight  machines  that  can 
be  devoted  to  programming  use  by  the  students  during  their  class  periods,  because  there  are  other  PC  compatibles  more 
readily  accessible,  and  because  it  offered  a  slightly  more  uniform  screen  and  keyboard  interface  to  the  programming  systems 
used.  The  lab  was  reserved  for  Uiese  students  duiing  their  class  times  and  was  accessible  to  them  for  much  of  (he  day  (when 
they  enjoyed  an  elevated  priority  of  access,  as  comparctl  with  students  that  wished  to  use  the  machines  for,  say,  word 
processing).  The  PCs  that  we  used  were  80386-basecl,  25  MH/.,  wiUi  I  MB  to  4  MB  RAM,  usually  a  40  MB  hard  drive, 
VGA  colour  monitor,  and  mouse. 

The  programming  sy.stcms  used  focus.scd  on  Turing  [Holt  92|,  LogoWriter  [LCSI  86),  Wisdom  Prolog  [MIT  Press  91],  and 
Smalltalk  /V  286  [Digitalk  88|.  Qujcn's  University  has  a  site  licence  for  Turing  (as  it  is  used  in  undergraduate  computer 
science  courses),  so  it  was  available  at  no  cost  to  the  Faculty  of  Education.  It  also  offered  a  low  cost,  full-featured  version 
(including  manuals)  to  students  who  order  the  product  (CDNS40.00).  LogoWriter  is  also  site-licensed  at  the  Faculty  of 
Education,  and  the  agreement  allows  for  student  use  at  home.  We  keep  LogoWriter  manuals  on  site.  We  chose  Wisdom 
Prolog  because  of  its  low  price  and  its  compliance  with  the  Edinburgh  standard.  Its  documenuuion  is  totally  inadequate  for 
Allen  Brown.  Queen's  Uinvcniiy 
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anything  more  than  syiHax  dcfiiiiliofis  and  special  fealurcs,  so  we  have  made  sure  the  library  has  several  books  of  varying 
complexity  on  Prolog.  Smalltalk  /V  286  was  chcsen  because  it  is  an  inexpensive,  full-featured  implementation  of  ihe 
Smalltalk  language.  It  is  specific  to  the  PC  environment,  recjuires  extended  memory,  and  it  is  different  from  the  original 
SmaIltalk-80  standard  [Goldberg  84J.  However,  it  runs  on  the  machines  that  we  have  (it  is  tight  on  machines  with  only  1 
MB  of  extended  memory).  A  DOS  version  exists  at  half  the  price,  but  its  debugger  and  features  are  not  nearly  as  good.  The 
Windows  version  is  too  costly  for  us.  Educational  institutions  receive  a  60%  discounted  price  when  ordered  directly  from 
Digitalk,  Inc. 

Reactions 

Student  reactions  have  generally  been  positive.  One  frequent  comment  was  that  the  course  was  a  lot  of  work.  Many 
students  found  the  pace  to  be  just  about  the  limit  of  what  they  could  coix;  with,  and  appreciated  that  an  in-depth  knowledge 
of  the  programming  system  was  not  required.  The  transition  from  one  language  to  the  next  caused  the  usual  discomfort  in 
just  learning  to  get  around  the  system;  the  editor,  file  saving,  and  so  on,  was  different.  The  shift  to  a  new  paradigm  was 
more  jarring.  StudeiUs  had  to  step  back  from  the  way  they  had  been  programming  for  the  past  few  hours  to  critically 
evaluate  what  was  different  about  what  they  were  now  doing.  This  causal  very  interesting  and  penetrating  insights  about 
how  a  programmer  tries  to  capture  a  particular  problem  in  a  programming  system.  They  questioned  the  differences  in  the 
geometry  bases  of  graphics,  why  sounds  were  generated  with  dilTerent  syniiixes,  why  declarations  were  important  in  one 
language  and  not  in  another,  why  some  systems  were  good  at  number  crunching  and  others  were  not,  how  data  could  be 
stored  and  why  (e.g.,  lists,  arrays,  collections),  and  generally  how  program  How  of  control  was  different 

The  most  significant  evidence  of  this  thinking  was  that,  in  selecting  their  projects,  most  students  first  analyzed  the  problems 
and  what  must  be  done  to  solve  them,  and  then  .sought  a  programing  system  that  would  offer  them  the  best  fit  to  their 
conceptual  representations.  For  many,  iliis  meant  learning  yet  another  programming  system.  Few  were  bothered  by  tliis,  as 
they  had  just  had  the  experience  of  sampling  four  systems  in  six  weeks!  In  several  cases  the  choice  was  related  to  their 
professional  interests,  and  il)ey  used  a  multimedia  system  such  as  HyperCard  or  Toolbook,  A  few  were  excited  by  the 
possibilities  of  languages  that  were  unfamiliar  to  them  (Prolog  and  Smalhalk)  and  chose  to  explore  these  more,  creating  a 
problem  that  would  be  suited  to  their  abilities  and  tliose  of  the  system.  Finally,  several  students  decided  they  could  make 
better  progress  using  more  conventional  systems  (Turing  and  LogoWriter)  and  chose  problems  that  were  appropriate  to 
them. 

Some  students  felt  intimidated  by  the  presence  ol*  colleagues  that  were  more  experienced  programmers  than  ihey  were. 
More  often  than  not,  those  with  more  expertise  were  peer  teachers  for  the  group.  Paradoxically,  only  a  few  of  these  pre- 
servicc  teachers  recognized  that  such  diverse  hctcr()gcna)us  groups  would  exist  in  their  own  classes  in  the  schools,  that  peer 
teaching  would  be  an  important  and  virtually  uncontrollable  as.set  in  them,  and  that  they  would  frequently  encounter 
students  with  more  knowledge  and  expertise  than  their  own  witliin  some  narrow  area.  This  has  much  to  do  with  their 
image  of  a  teacher  as  a  font  of  knowledge,  a  dispenser  of  wisdom,  a  leader  of  a  harmonious  band.  It  was  also  interesting  to 
see  their  reactions  to  the  rapid  overview  provided  by  such  a  concepij:ich  and  technically-oriented  subject  area.  Some  loved 
it,  others  felt  constantly  on  the  alge  of  confusion.  Again,  few  students  recognized  that  this  would  be  the  range  of  reactions 
to  much  of  the  material  that  they  would  be  teaching,  whatever  the  subject  area. 

The  project  wiUi  a  particularly  heavy  workload,  mamly  bccau.sc  tlte  students  set  much  loftier  goals  and  standards  than  the 
instructor.  The  students  were  advised  on  how  to  prototype  their  work  and  develop  it  incrementally.  In  doing  so  the  projects 
were  more  manageable  and  "jrew"  to  a  state  of  completeness  iluit  everyone  could  be  satisfied  with. 

The  instructor  was  pleased  with  outcomes  of  the  course.  Some  students  who  returned  a  year  later  said  that  ihey  had  worked 
very  hard  in  the  course  and  that  the  real  knowledge  of  what  they  had  learned  was  just  beginning  to  dawn  on  them.  Students 
exchanged  programs  after  the  presentation  and  felt  genuine  pleasure  and  pride  in  what  they  had  accomplished. 
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Future  Directions 

There  will  be  a  formal  15  ininulc  iniroduciion  al  ihc  slarl  of  each  new  paradigm.  This  will  include  an  anlhropomorphized 
example  of  giving  inslruciions  ihai  must  be  followed  according  lo  a  preset  pailern.  This  should  set  the  tone  for  a  shift 
(rather,  a  break)  in  the  thinking  modes  needed  for  the  next  paradigm.  The  class  notes  will  continue  to  be  revised,  to  reflect 
greater  emphasis  on  graphics,  sound,  and  possibly  muliifncdia,  which  are  underutilized  dimensions  of  programming  at 
present.  The  instructor  will  also  informally  emphasi/x  the  types  of  learning  that  are  occurring  in  the  class  and  ask  students 
to  reflect  on  these  observations  in  the  context  of  their  own  leaching  and  learning. 

Although  the  atmosphere  is  quite  convivial  during  the  project  presentations,  a  concurrent  pixza  party  is  also  planned! 
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Abstract 

This  paper  describes  the  process  of  development  of  courseware  for  teaching  compilers  in 
Computer  Science  programme  using  Quest  (Allen  Communication)  authoring 
system.The  compiler  course  is  particularly  difficult  to  teach,  because  it  uses  '^ny 
theoretical  and  practical  concepts.  For  this  reason,  the  computer-based  presentati^/j  is 
ideal  for  this  course.  We  demonstrate  how  these  essential  features  could  be  implemented 
in  Quest. 


Introduction 

Early  in  the  history  of  computers,  their  potential  for  instructional  purposes  has  been 
recognized.  In  the  1960's  and  1970's  much  research  in  this  area  was  done,  but  the 
practical  results  were  limited  by  the  hardware  available.  Since  the  development  of 
affordable  and  fast  personal  computers  in  the  1980's  and  1990's,  the  situation  has 
changed  dramatically. 

The  oldest,  and  yet  still  most  widely  used  medium  for  the  presentation  of 
information  is  the  book.  Since  CBT  [Ale91]  is  often  based  on  a  book  metaphor, 
cour'^.eware  should  be  at  iaast  as  good  as  the  conventional  book.  To  this  end,  courseware 
should  support  the  following  tools,  which  are  available  to  anyone  reading  a  book.  The 
user  should  be  able  to  move  to  any  page  at  any  time  and  start  reading.  The  position  in  the 
courseware  of  the  reader  should  be  apparent.  Some  type  of  notetal<ing  capability,  either 
global  or  margin  notes,  and  highlighting  should  also  be  supported.  A  table  of  contents  and 
index  are  also  necessary.  Courseware  also  should  make  use  of  text  and  graphics. 
Besides  these  standard  book  features,  courseware  has  the  capability  to  go  beyond  the 
book  by  providing  more  advanced  tools.  Perhaps,  the  most  important  capability  is  the  use 
of  hypertext.  Hypertext  [Woo91]  is  text  organized  as  a  directed  graph,  rather  than  a 
linear  sequence  of  phrases.  It  permits  the  rapid  and  efficient  access  of  further  relevant 
information.  In  addition,  the  power  of  ihe  computer  can  be  used  in  courseware  to  provide 
for  animation  and  simulation.  These  two  tools  are  poweiful  visualization  techniques  that 
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go  beyond  the  capabilities  of  a  book.  Finally,  courseware  should  interact  W\\h  the  student. 
The  student  can  be  asked  questions,  and  the  courseware  should  be  able  to  analyze  the 
response  and  judge  if  it  is  correct.  Immediate  feedback  on  these  questions  helps  the 
learner  discover  which  areas  require  more  study.  One  last  feature  that  an  authoring 
system  should  also  provide  for  is  support  of  student  management,  or  computer  managed 
instruction  (CMI).  CMI  is  used  for  the  maintenance  of  student  records.  Student  records 
should  be  available  based  on  performance  criteria,  the  number  of  courses  completed,  and 
others.  Registration  of  students  to  specific  courses  and  keeping  separate  records  for 
each  student  registered  should  be  available. 

The  development  of  books  into  their  present  form  has  taken  over  thousands  of 
years,  whereas  courseware  is  still  developing  and  has  many  issues  that  remain  to  bo 
resolved.  The  most  important  issue  revolves  around  the  instructional  strategy  used.  Most 
current  courseware  is  based  on  the  Branching  Programmed  Instruction  Model  [Mer85]. 
This  model  organizes  instruction  Into  the  following  sequence  of  events:  students  are 
presented  with  information,  asked  a  question,  'eedback  is  provided  based  on  the 
student's  response,  and  then  the  cycle  is  repeated.  There  is  much  debate  over  whether 
this  strategy  is  sufficient^  or  if  another  should  be  used. 

There  are  three  main  ways  to  create  courseware.  A  general  purpose  programming 
language,  like  Pascal  [Coo87]  or  C  [Ker78],  can  be  used.  This  method  requires 
experienced  programmers  and  considerable  development  time.  Using  a  general  purpose 
language  is  very  flexible,  however,  and  provides  the  best  speed  of  execution.  Second,  a 
special  purpose  authoring  language  can  be  used.  An  authoring  language  is  simply  a 
programming  language  with  additional  functions  that  make  the  creation  of  courseware 
simpler.  This  has  advantages  over  the  first  approach  in  that  the  creation  of  courseware  is 
easier  because  of  the  added  functionality  of  the  language.  Its  main  disadvantage  is  its 
complexity.  For  each  new  function  added  to  provide  for  courseware  development,  the 
author  must  learn  how  to  use  this  function.  The  third  method  is  the  authoring  system.  An 
authoring  system  has  three  main  goals:  minimize  or  enminate  programming,  provide 
student  management  support,  and  include  instructional  templates.  Unfortunately, 
authoring  systems  are  inflexible  for  the  author  who  wants  to  perform  a  task  outside  the 
scope  of  the  system.  Most  authoring  systems  use  the  Branching  Programmed  Instruction 
Model  for  instruction,  and  they  lock  the  author  into  using  this  model. 

The  primary  goal  of  this  paper  was  to  implement  courseware  for  teaching 
compilers  in  Computer  Science  programme.  This  course  is  particularly  difficult  to  teach, 
because  it  uses  many  theoretical  concepts  and  also  requires  a  considerable  practical 
experience.  Thus,  the  teacher  has  to  often  refer  to  concepts  introduced  eariier  in  the 
course,  or  covered  in  other  courses.  At  the  same  time,  the  teacher  has  to  show 
simulations  of  various  tools,  such  as  finite  automata,  scanners,  parsers,  etc.  For  these 
reasons,  a  computer-based  presentation  is  ideal  for  the  compiler  course. 

V/e  assumed  that  the  created  courseware  should  have  all  the  essential  tools 
described  above.  We  also  took  into  consideration  the  fact  that  courseware  for  computer 
science  is  designed  for  a  special  audience,  namely?  for  the  computer  science  students. 
We  decided  to  use  an  MS-DOS  based  authoring  system  and  choso  Quest  (Quest  is  a 
registered  trademark  of  Allen  Communication,  Inc.). 
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Our  paper  is  organized  as  follows.  In  Section  2,  we  describe  requirements  for 
courseware  for  teaching  compilers.  Section  3  provides  a  short  introduction  to  Quest. 
Section  4  describes  the  compiler  courseware  that  has  been  implemented,  and  Section  5 
is  on  implementation  of  this  courseware.  In  particular,  we  discuss  the  implementation  of 
various  features  for  which  Quest  is  not  really  suitable;  for  example  multiple  windows 
and  support  for  hypertext. 

We  assume  that  the  reader  is  familiar  with  the  basic  terminology.  For  more  details 
of  our  project,  see  [Low92]. 


2  Requirements  for  Courseware  for  Teaching  Compilers 

Designing  courseware  typically  requires  expertise  in  at  least  two  fields.  One  expert  is 
required  in  the  field  specific  to  the  course  being  created  and  another  is  required  to 
actually  create  the  courseware.  This  second  expert  is  usually  a  computer  programmer. 
One  of  the  goals  of  authoring  systems  is  to  eliminate  the  need  for  the  computer  expert  by 
allowing  a  person  to  easily  create  courseware,  without  any  specialized  computer  skill. 
Authoring  systems  hide  the  internals  of  the  creation  process  by  allowing  the  designer  to 
work  at  a  very  high  level  of  abstraction.  Unfortunately,  the  authoring  system  may  not 
provide  all  the  tools  the  author  requires,  and  there  is  little  recourse  for  the  author  who 
wishes  to  perform  some  task  that  the  authoring  system  does  not  support. 

When  courseware  is  being  created  for  computer  science,  the  course  expert  is  a 
computer  expert.  The  computer  science  author  who  is  familiar  with  programming  is  not 
restricted  to  the  highest  level  of  the  authoring  system  but  can  choose  to  switch  to  a  low 
level,  for  example,  programming  in  the  C  programming  language,  if  required.  In  addition, 
the  author  of  computer  science  courseware  can  benefit  greatly  from  the  level  of  the 
students.  For  example,  it  can  be  safely  assumed  that  the  majority  of  the  students  will  be 
familiar  with  computers,  their  basic  operation,  and  terminology  used.  Such  knowledge  can 
free  the  author  from  having  to  give  lengthy  tutorials  on  the  basics.  It  should  be  noted, 
however,  that  computer  science  students  are  demanding  users  and  therefore  courseware 
for  them  must  be  particularly  well  designed  and  implemented. 

Besides  the  standard  support  required  for  computer  based  teaching,  courseware 
for  computer  science  has  some  specialized  requirements.  Perhaps  the  most  important  of 
these  is  communication  with  outside  programs  [Mul91].  The  student  should  not  be 
required  to  exit  the  courseware  to  conduct  experiments  or  to  use  examples.  An  excellent 
example  of  the  need  for  communication  is  courseware  for  teaching  programming 
languages.  In  order  to  provide  for  the  best  teaching  environment,  a  link  to  the  appropriate 
compiler  is  necessary  to  allow  the  student  to  compile  and  execute  examples  while 
remaining  within  the  courseware.  There  are  different  methods  for  this  communication.  The 
most  primitive  one  is  through  the  use  of  files.  One  program  can  leave  information  in  a  file, 
where  another  can  read  it.  File  communication  is  available  regardless  of  the  environment, 
but  it  is  slow  and  cumbersome.  A  better  solution  is  to  use  available  tools  within  the 
environment.  Programs  running  under  Microsoft  Windows  (Windows  is  a  trademark  of 
Microsoft  Corporation)  can  communicate  with  each  other  through  Dynamic  Data 
Exchange  (DDE)  [Nor92].  This  type  of  interprocess  communication  is  a  significant 
improvement  over  the  use  of  files,  but  it  is  not  always  available  due  to  the  requirement  of 
the  environment.  Microsoft  Windows  specific  programs  are  required  to  take  advantage  of 
DDE;  non-windows  programs  cannot  use  this  communication  method.  The  underlying 
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operating  system  and  environment  are  important  aspects  of  authoring  systems.  For  many 
authors,  MS-DOS  does  not  provide  enough  support  for  courseware.  In  the  case  of 
designing  courseware  for  teaching  compilers,  there  is  a  need  for  links  with  various 
compiler  tools  that  are  discussed,  for  example  LEX  and  YACC. 

Courseware  for  computer  science  also  requires  sophisticated  judging  facilities. 
Simple  judging  capabilities  like  word  or  number  matching,  which  are  present  in  most 
authoring  systems,  are  insufficient.  Analyzing  computer  programs  is  a  very  difficult  task 
and  frequently  involves  using  some  artificial  intelligence  techniques.  A  link  to  the  compiler 
is  useful  to  the  student  to  see  if  the  program  compiles,  but  program  correctness  cannot  be 
determined.  A  student's  program  may  compile,  but  this  does  not  ensure  that  the  program 
answers  the  question  correctly.  Determining  the  correctness  of  programs  still  requires  a 
human  marker.  A  fully  complete  courseware  package  for  computer  science  would  include 
this  capability. 

In  the  next  section,  we  briefly  introduce  Quest. 


3  Introduction  to  Quest 

Quest  is  an  MS-DOS  based  authoring  system  and  it  aims  to  provide  all  the  necessary 
tools  required  to  create  multimedia  presentations.  Since  Quest  runs  under  MS-DOS.  it 
suffers  from  the  communication  limitations  of  that  operating  system.  Quest  is  frame 
based,  meaning  that  each  lesson  consists  of  a  number  of  frames  that  may  be  considered 
as  pages  in  a  book.  Each  screen  of  information  presented  to  the  student  on  the  monitor  is 
one  frame  of  the  lesson.  Frames  can  be  easily  constructed  as  Quest  provides  several 
different  types  of  objects  that  can  be  placed  in  frames.  Many  of  these  objects  are  similar 
to  the  objects  that  may  be  found  in  a  drawing  package,  such  as  text,  circles,  and 
rectangles.  Others  are  more  sophisticated  such  as  branches,  audio,  video,  and  answer 
analysis  (judging).  Frames  are  entirely  composed  of  these  objects.  All  of  these  objects 
enable  the  author  to  present  information  and  to  question  the  student  without  any  specific 
knowledge  of  programming.  In  or(*er  to  create  complex  effects  and  operations  however, 
the  author  must  resort  to  writing  programs. 

There  are  many  different  types  of  Quest  objects;  they  are  separated  into  the 
following  groups:  text,  graphics,  shape,  image,  display,  performance,  branch,  event, 
video,  audio,  conditional  execution,  Quest  Authoring  Language  (QAL)  program,  and 
group.  Branch  objects  allow  the  author  to  connect  frames.  There  are  different  types  of 
branches,  for  example  Unconditional  branch  (UBR).  similar  to  a  goto  statement  in  Basic. 
Quest's  performance  object  gives  authors  a  wide  set  of  answer  analysis  or  judging 
capabilities.  The  author  can  supply  varied  and  direct  feedback  based  on  the  answer  the 
student  provided. 

An  aspect  of  Quest  that  allows  the  author  to  go  beyond  the  usual  functionality  of 
an  authoring  system  is  QAL  objects.  QAL  is  a  Pascal-like  programming  language  with  a 
large  set  of  these  functions  that  mirror  the  capabilities  of  the  other  objects.  For  example, 
there  are  graphics  functions  to  draw  circles  and  squares,  as  well  as  to  change  fonts  or 
colors.  Although  QAL  provides  these  functions,  they  are  easier  to  implement  using  more 
appropriate  objects.  For  instance,  it  is  simpler  to  create  a  rectangU  using  a  rectangle 
object  rather  than  to  write  a  QAL  program.  The  advantage  of  QAL  lies  in  the  functions  it 
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provides  that  cannot  be  performed  with  the  other  objects.  For  example,  QAL  provides  for 
file  I/O,  as  well  as  mouse  control.  Also  available  are  memory  functions  such  as  peek  and 
poke  and  a  BIOS/DOS  interrupt  interface.  One  example  of  the  use  of  QAL  is  to  combine 
mouse  and  keyboard  input.  QAL  is  not  the  only  way  the  author  can  integrate  a  program 
into  the  courseware.  Using  a  Call  Program  branch.  (PGM)  Quest  can  call  other 
programs,  which  may  be  written  in  another  language.  The  author  could  write  a  program  in 
C,  or  any  other  language,  and  then  use  a  PGM  branch  to  run  it.  The  advantage  of  using 
QAL  instead  of  other  languages  is  that  QAL  can  communicate  with  Quest  through 
global  variables,  while  programs  written  in  other  languages  cannot.  In  addition,  a  QAL 
object  is  actually  inserted  into  the  frame.  This  means  that  the  QAL  program  need  not  be 
physically  present  on  the  disk  when  the  student  uses  the  courseware.  This  is  not  the  case 
when  a  PGM  branch  is  used,  because  in  that  case  the  program  must  be  present  on  the 
disk.  The  disadvantage  of  QAL  is  its  simplistic  structure,  without  such  features  as 
recursion,  external  procedures,  or  separate  compilation. 


4  Description  of  the  Compiler  Courseware 

To  design  our  courseware,  the  test  and  exam  questions  from  previous  courses  on 
compilers  taught  at  Acadia  University  were  categorized,  and  it  was  determined  that  most 
of  these  questions  were  based  on  the  areas  specified  below.  The  other  possible  lessons; 
for  example  on  type  checking  were  left  out,  due  to  time  constraints. 

The  course  is  divided  into  the  following  lessons  and  sections: 

•  Introduction 

♦  Using  the  System 

♦  Introduction  to  Compilers 

•  Lexical  Analysis 

♦  Regular  Definitions 

♦  Transition  Diagrams 

♦  LEX 

•  Syntax  Analysis 

♦  Context  Free  Grammars 

♦  Recursive  Descent  Parsers 

♦  Table  Driven  Parsers 

♦  Shift  Reduce  Parsers 

♦  YACC 

The  Using  tho  System  section  provides  students  with  information  on  how  to 
navigate  through  the  courseware  and  conventions  used,  for  example  the  use  of  hotwords. 
The  Introduction  to  Compilers  section  is  a  brief  overview  of  the  entire  course. 

The  Regular  Definition  section  presents  the  folk^wing  terms:  alphabet,  language, 
regular  expressions,  and  regular  definitions*  It  also  explains  regular  expressions  and 
regular  definitions  and  how  tokens  are  represented  using  regular  definitions.  The  section 
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Transition  Diagrams  demonstrates  how  to  create  transition  diagrams  from  regular 
definitions  and  the  code  for  the  scanner  from  these  diagrams.  The  LEX  section  gives  an 
overview  of  how  to  use  LEX  and  some  of  its  important  features.  This  section  makes 
considerable  use  of  examples.  Regular  expressions  and  definitions  are  used  extensively 
in  LEX,  so  ideas  presented  in  the  first  section  are  used. 

Context  Free  Grammars  explains  the  use  of  CFG's  in  describing  a  grammar  and 
introduces  some  important  terminology.  Recursive  Descent  Parsers  presents  a  recursive 
implementation  of  top-down  parsing.  It  illustrates  the  steps  necessary  to  constructing  a 
recursive  descent  parser,  including  left  factoring  and  eliminating  left  recursion.  An 
interactive  example  of  the  latter  is  given.  Table  Driven  Parsers  presents  a  non-recursive 
implementation  of  top-down  parsing.  It  explains  the  construction  of  the  sets  FIRST  and 
FOLLOW,  as  well  as  the  parsing  table.  This  lesson  also  presents  an  interactive  example 
on  the  construction  of  both  sets  and  the  table.  The  concept  of  a  LL(1)  grammar  is 
introduced.  Another  interactive  example  of  the  parsing  algorithm  is  then  shown,  using  the 
previously  generated  table.  The  section  Shift  Reduce  Parsers  explains  the  difference 
between  top-down  and  bottom-up  parsing  and  then  proceeds  through  a  lengthy 
interactive  example  of  parsing  a  string  given  the  algorithm  and  parsing  table.  The  section 
on  YACC  (a  bottom-up  parser  generator  [Aho86])  gives  the  basics  for  writing  YACC 
programs  and  provides  several  examples.  It  demonstrates  how  to  use  LEX  and  YACC 
together  to  create  a  complete  front  end  of  a  compiler. 

In  our  couroeware,  students  are  not  required  to  follow  any  fixed  path;  they  can 
jump  between  lessons  and  sections  at  wilL  Students  also  have  control  over  questions. 
The  courseware  does  not  force  them  to  answer  questions  correctly  or  at  all  nor  does  it 
keep  a  score  of  correct  or  incorrect  answers.  This  type  of  student  control  was  used 
phmarily  due  to  the  nature  and  level  of  the  students  (the  upper  level  university  students). 
Another  reason  for  this  design  was  the  goal  of  the  courseware,  it  is  not  meant  to  be  a 
replacement  to  the  actual  course  but  rather  as  an  additional  tool  for  students  to  use.  It  is 
designed  to  be  a  better  book.  At  the  beginning  of  the  course  and  each  lesson,  a  menu  is 
displayed  allowing  the  student  to  make  choices  on  where  to  begin  study.  The  student  can 
make  choices  using  either  the  mouse  or  the  keyboard. 

Question  frames  are  meant  to  drill  the  student  on  the  topics  previously  covered  in 
the  tutor  frames.  The  purpose  of  question  frames  is  to  reinforce  knowledge  through 
interaction  the  recent  information  assimilated.  There  is  no  score  kept  for  incorrect  or 
correct  answers,  and  the  student  is  not  required  to  answer  correctly  or  even  answer  at  all. 
The  student  is  immediately  informed  about  correct  and  incorrect  answers  and  is  also 
allowed  the  opportunity  to  continue  answering,  as  well  as  to  see  the  answer.  Another  type 
of  frame  is  the  interactive  example  frame.  The  student  can  move  back  and  forth  through 
the  example,  easily  seeing  exactly  what  is  taking  place  at  each  step. 

Window  frames  are  used  to  implement  some  of  the  important  functions  such  as  the 
index,  margin  notes,  and  help.  The  windows  appear  on  top  of  the  current  screen  and 
restore  the  original  screen  when  they  disappear.  All  the  window  frames  can  be  dismissed 
by  pressing  the  Esc  key  or  by  clicking  with  the  mouse  in  the  upper  left  corner  of  the 
window.  This  area  of  the  window  is  called  the  window  dismiss  box. 
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Students  may  encounter,  on  certain  frames,  words  or  phrases  that  are  displayed 
as  blue  text  on  a  white  background.  These  are  known  as  hotwords.  The  student  can  click 
on  hotword§  with  the  mouse  to  obtain  more  information,  so  these  words  form  electronic 
links  to  this  information,  in  some  cases,  a  list  of  pages  where  there  is  additional 
information  related  to  the  topic  is  available.  The  student  can  jump  to  a  listed  page  by 
selecting  it. 

The  student  is  provided  with  a  variety  of  essential  tools  that  can  be  invoked  via 
icons  located  along  the  bottom  of  the  screen.  These  tools  implement  the  important 
functions  necessary  for  courseware:  global  notes,  index,  goto  with  table  of  contents,  page 
history,  linear  traversal  {backward  and  forward),  and  exiting.  Icons  can  be  selected  with 
the  mouse  or  the  keyboard,  using  a  special  key  combination.  They  are  available  at  all 
times  at  all  frames,  except  the  Using  the  System  section  where  the  icons  are  introduced. 
Below,  we  list  all  the  essential  icons: 

•  Quit  icon  quits  the  student  entirely  from  the  course  and  saves  the  frame  where  the 
student  is  in  the  lesson.  When  the  same  course  is  entered,  Quest  will  restore  the 
student  to  that  frame, 

•  Escape  icon  takes  the  student  back  to  the  nearest  menu  screen.  At  the  top  level  menu 
screen.  Escape  will  exit  the  student  from  the  courseware  in  the  same  manner  as  the 
Quit  icon, 

•  Shell  to  DOS  icon  allows  the  student  to  spawn  a  command  shell.  The  student  can 
return  to  the  same  page  where  the  shell  was  started  by  typing  exit  at  the  DOS  prompt. 

•  Goto  icon  opens  a  pop-up  window  which  allows  the  student  to  enter  a  page  to  jump 
to.  Alternatively  the  student  can  click  on  the  table  button,  which  will  present  the  initial 
table  of  contents  window.  The  table  of  contents  consists  of  several  windows  which 
present  the  student  with  a  complete  list  of  the  lessons,  sections,  and  individual  pages 
within  the  course.  The  first  window  contains  a  list  of  the  available  lessons  from  which 
the  student  can  select  a  lesson.  Then  a  window  with  a  list  of  the  sections  within  that 
lesson  appears.  By  selecting  a  section  from  this  list,  the  student  will  activate  a  third 
window,  which  is  a  list  of  the  individual  pages  in  the  selected  section.  If  the  student 
then  selects  a  page,  a  jump  is  performed  to  that  page.  .At  any  time  the  student  can 
press  the  ESC  key  or  click  in  the  window  dismiss  box  in  the  upper  right  corner  to 
dismiss  a  window  and  return  to  the  previous  one. 

•  Back  icon  gives  access  to  the  page  history,  a  list  of  the  previously  visited  pages.  A 
student  can  cycle  through  this  list  by  repeatedly  invoking  this  icon.  Using  the  icon 
once  will  move  the  student  to  the  first  page  in  the  history;  each  successive  usage  will 
move  to  the  next  page  in  the  history  until  the  history  is  exhausted.  To  go  back  from  the 
keyboard,  the  student  can  use  the  home  key. 

•  Book  icon  allows  the  student  to  invoke  a  text  editor  for  the  purpose  of  taking  notes. 
These  notes  are  saved  in  the  student's  own  file.  Notes  are  global,  that  is,  they  are  not 
associated  in  any  way  with  the  current  frame  but  with  the  entire  courseware.  Margin 
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notes,  associated  with  the  current  frame,  have  also  been  implemented  although  they 
have  not  been  fully  integrated  into  the  courseware  due  to  time  constraints. 

•  Index  icon  presents  the  student  with  the  user  modifiable  index  (see  Figure  4).  The 
student  can  delete,  enter,  and  modify  entries  in  the  index.  All  students  have  their  own 
index;  modifications  one  student  makes  will  not  affect  the  index  of  another  The 
deletion  and  modification  of  index  entries  are  done  by  first  highlighting  the  desired 
entry  and  then  clicking  on  the  appropriate  button.  Students  can  also  use  the  Delete 
key  on  the  keyboard  to  delete  entries.  The  student  can  insert  entries  by  clicking  on  the 
insert  button  and  then  typing  in  the  desired  entry. 

•  Previous  and  Next  icons  return  respectively  to  the  previous  and  the  next  page. 


5  Impiementation 

The  implementation  was  influenced  by  the  design  decisions  made  during  the  initial  design 
phase  of  the  compiler  courseware.  The  main  decision  was  to  combine  keyboard  and 
mouse  input  as  much  as  possible.  This  resulted  in  the  eventual  decision  to  use  QAL  as 
the  main  implementation  tool.  For  this  reason,  much  of  the  implementation  is  non- 
standard, and  QAL  programming  was  unavoidable.  In  particular,  tools  like  the  modifiable 
index,  require  QAL. 

Global  variables  provide  for  communication  between  Quest  and  QAL  programs 
and  are  available  in  Quest  by  using  embedded  statements.  Embedded  statements  are 
simply  statements  in  text  objects,  placed  in  frames.  Since  frames  can  only  communicate 
through  global  variables,  a  large  number  of  them  are  used  in  the  compiler  courseware.  In 
particular,  global  variables  were  used  to  implement  the  index  and  history  feature.  For  the 
entire  index  facility  there  are  four  separate  frames.  The  main  index  frame  reads  the  entire 
student  index  file  into  memory,  storing  it  in  a  global  variable.  The  add,  modify,  and  goto 
frames  can  access  this  global  variable  and  perform  their  actions  on  the  data  in  memory 
rather  than  in  the  file.  Without  the  global  variable,  these  frames  would  have  to  either  read 
the  file  into  their  own  local  variable  or  be  included  into  a  large  QAL  program  in  the  main 
frame.  Another  use  for  the  global  variables  is  the  page  history,  which  is  stored  in  memory 
as  a  global  array  of  strings. 

Hypertext  is  an  important  and  powerful  tool  for  courseware.  Quest  does  not 
support  in  any  way  the  use  of  hypertext,  and  this  is  a  serious  drawback.  The  author  must 
resort  to  using  QAL  to  implement  links,  a  difficult  and  time  consuming  task.  In  order  to 
follow  an  electronic  link,  the  student  is  usually  required  to  click  on  a  certain  screen 
region.  Each  link,  therefore,  must  be  manually  coded  in  QAL,  with  the  author  responsible 
for  maintaining  the  pixel  region  of  the  link.  Any  change  to  the  structure  of  the  frame,  such 
as  moving  text,  requires  the  author  to  update  the  pixel  area. 

Although  using  global  variables  to  reduce  the  number  of  programs  can  be  used  in 
many  instances,  separate  programs  are  still  needed.  For  example,  the  continuous 
animation  in  the  introduction  to  the  Lexical  Analysis  lesson  requires  a  separate  program. 
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Navigation  in  Quest  courseware  can  be  done  in  several  ways.  The  author  can 
assign  special  student  keys  to  forward  and  backward  movement.  This  method  was 
determined  to  be  too  restrictive  for  the  compiler  courseware,  mainly  because  of  the  lack 
of  mouse  support  for  many  of  the  functions.  All  the  icons  could  have  been  implemented 
using  the  student  keys,  but  the  students  would  have  been  restricted  to  mostly  keyboard 
input.  Since  mouse  input  v/as  available  only  for  a  small  subset  of  all  the  functions,  each 
frame  in  the  compiler  courseware  has  a  QAL  object  followed  by  a  performance  object  as 
the  last  objects  in  the  frame.  The  QAL  program  polls  the  keyboard  and  mouse,  waiting  for 
events.  When  the  student  performs  an  action  that  requires  a  branch,  the  QAL  program 
sets  the  answer  variable  to  the  appropriate  string.  For  example,  clicking  the  quit  icon  sets 
the  answer  variable  to  the  string  -quif.  The  performance  object  then  performs  an  answer 
analysis  and  executes  a  specific  branch  based  on  what  the  answer  variable  is,  A 
performance  object  is  necessary  because  many  different  branches  are  needed.  For 
example,  Quest  treats  branches  within  a  lesson  as  different  from  branches  to  other 
lessons,  so  anytime  there  is  a  branch  from  one  lesson  to  another  a  different  branching 
object  must  be  used.  Another  method  to  branch  to  different  frames  is  to  use  the 
Expression  branch.  This  branch  is  inadequate  because  it  does  not  allow  branches  to 
other  lessons. 

Every  frame  in  the  compiler  courseware  shares  the  same  structure  and  is  specified 
as  a  no-erase  frame.  The  frame  first  displays  its  information,  then  it  waits  for  student 
input.  Since  the  student  input  expected  is  the  same  for  most  frames,  once  the  frame 
information  is  displayed,  an  automatic  branch  is  executed  to  a  common  image  frame.  This 
image  frame  contains  only  the  QAL  and  performance  objects;  it  does  not  display  anything. 
Each  available  function  has  a  corresponding  answer  and  performance  sequence.  Adding 
more  tools  to  the  courseware  can  easily  be  done  by  minor  modifications  to  the 
performance  object  and  the  QAL  program  (by  adding  another  answer).  Since  these 
objects  are  in  only  one  frame  and  used  by  all  the  other  frames,  the  changes  are  quickly 
integrated.  It  has  been  noted  that  some  frames  require  special  QAL  programs;  these 
frames  also  require  a  special  performance  object  and  so  cannot  use  the  global  image 
frame.  The  use  of  image  frames  serves  two  purposes:  it  keeps  the  implementation  of  the 
functions  in  a  single  place  making  modifications  easier  and  it  also  increases  execution 
speed.  If  the  student  selects  a  function  that  displays  a  window,  this  window  saves  the 
screen  area  where  it  is  displayed  and  then  restores  the  screen  area  when  dismissed.  If 
the  window  returned  to  the  original  frame,  the  entire  information  of  that  frv^ime  would  be 
redisplayed  unnecessarily.  Instead,  all  the  window  frames  return  to  the  image  frame  when 
dismissed.  Since  nothing  is  drawn  on  the  screen  in  the  image  frame,  this  speeds  up 
execution  of  the  courseware. 

Mouse  operations  were  implemented  using  QAL.  The  usual  method  of  controlling  a 
mouse  when  using  a  programming  language  is  with  MS-DOS  interrupt  calls.  This  was  not 
necessary  with  QAL  however,  since  it  provides  for  basic  mouse  operation  through  special 
functions.  Using  QAL  enabled  complete  control  over  mouse  functions.  For  example, 
mouse  button  clicks  from  any  button  could  be  trapped.  The  drawback  to  this  method  is 
that  the  program  is  now  responsible  for  all  mouse  operations.  Quest  cannot  be  used  to 
control  any  of  the  functions  of  the  mouse.  One  of  the  implications  of  this  requirement  is 
that  absolute  pixel  coordinates  must  be  checked  whenever  the  mouse  button  is  clicked. 
The  program  continually  pells  the  keyboard  and  mouse  waiting  for  events.  When  a  mouse 


click  occurs,  the  coordinates  of  the  mouse  pointer  are  compared  with  a  coded  list  of 
screen  areas.  Each  of  the  icons  encompasses  a  rectangular  region  of  the  screen.  For 
every  icon,  the  position  of  each  of  the  four  corners  is  known  and  is  coded  into  the 
program.  This  is  a  rather  difficult  and  error  prone  approach.  For  example,  let  us  examine 
what  happens  when  a  student  clicks  on  an  icon.  When  the  mouse  is  clicked,  the  QAL 
program  then  obtains  the  mouse  pointer  coordinates  in  absolute  pixels.  The  coordinates 
of  the  mouse  are  then  compared  to  see  if  they  are  within  the  boundaries  of  one  of  the 
icons.  !f  the  mouse  pointer  was  not  within  any  of  the  icon  boxes,  the  program  ignores  the 
mouse  event  and  continues.  However,  if  the  pointer  was  within  the  box  delimited  by  an 
icon,  the  function  represented  by  that  icon  is  started.  Typically,  when  the  icon  is 
depressed,  the  answer  variable  is  set  to  the  appropriate  string  and  the  program  exits.  The 
performance  object  then  analyzes  the  answer  variable  and  performs  some  type  of  branch 
based  on  its  value.  The  buttons  and  icons  in  the  compiler  courseware  actually  move;  they 
depress  when  the  mouse  button  is  depressed  and  return  to  normal  when  the  button  is 
released.  This  three  dimensionai  effect  was  implemented  using  two  shapes  for  each 
button  or  icon:  an  up  shape  and  a  down  shape.  Originally  the  up  shape  is  displayed,  but 
when  the  student  clicks  on  the  button  or  icon,  the  down  shape  is  displayed  as  long  as  the 
mouse  button  is  depressed  and  the  pointer  is  within  the  icon  or  button  box.  When  the 
button  is  released,  or  the  mouse  pointer  is  moved  out  of  the  delimiting  box,  the  original 
shape  is  displayed.  All  of  these  shapes  are  stored  in  the  shape  library.  Another  approach 
to  signify  mouse  events  is  to  cut  an  image  of  the  screen  and  then  paste  it  back  in  reverse. 
This  approach  is  used  for  list  selections,  menu  selections,  and  the  window  dismiss  box. 
This  cutting  and  pasting  technique  is  also  an  excellent  way  to  highlight  areas.  The 
original  image  can  easily  be  restored  by  pasting  the  cut  image  in  normal  mode,  rather 
than  reverse. 

In  Quest  there  are  three  ways  to  use  mouse  input:  QAL,  position  answer 
analysis,  and  Quest  student  keys.  Of  these,  only  QAL  is  acceptable  for  all  our 
requirements.  By  using  the  position  answer  analysis  method,  screen  areas  can  be 
associated  with  certain  answers.  This  is  what  the  QAL  program  does;  mouse  clicks  in 
certain  screen  regions  cause  the  assignment  of  different  strings  to  the  answer  variable.  It 
would  seem,  therefore,  that  the  position  answer  analysis  approach  is  satisfactory,  but  it 
has  some  restrictions.  Its  most  significant  drawback  is  that  keyboard  input  is  not  accepted 
in  the  normal  sense.  The  arrow  keys  can  be  used  to  move  the  pointer  to  the  screen  areas; 
the  Enter  key  is  used  to  activate  the  area.  In  this  way  the  keyboard  is  used  *o  simulate  a 
mouse.  This  was  not  the  kind  of  keyboard  input  that  was  desired  because  it  is  very 
unnatural.  One  of  the  benefits  of  assigning  certain  key  combinations  to  functions  is  that 
these  key  combinations  are  typically  faster  to  use  than  the  mouse.  Simulating  the  mouse 
with  the  keyboard  actually  makes  keyboard  input  slower.  Another  disadvantage  is  that  the 
mouse  cursor  changes  to  a  crosshair  while  it  is  normally  an  arrow.  The  second  method,  of 
using  Quest  student  keys,  allows  for  the  normal  use  of  the  keyboard  and  the  mouse, 
overcoming  the  limitation  of  the  previous  technique,  but  it  also  is  not  sufficient.  In  addition 
to  being  able  to  assign  certain  keys  to  functions,  some  of  the  functions  can  also  be 
assigned  screen  areas.  The  main  disadvantage  is  that  these  functions  are  fixed  and  do 
not  provide  enough  capability  for  i'no  oompiler  courseware.  The  following  functions  may 
have  screen  areas  assigned:  moving  forward  and  backward,  exiting,  a  table  of  contents, 
help,  and  a  glossary.  For  other  functions,  such  as  hotwords  or  taking  notes,  a  mouse 
cannot  be  used.  A  significant  problem  that  plagues  both  solutions  is  that  the  function  Is 
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activated  when  the  right  button  is  pressed.  This  is  not  a  button  click,  since  the  generally 
accepted  definition  of  a  button  click  is  a  press  and  release.  Tne  other  buttons  on  the 
mouse  cannot  be  tracked  at  all. 

In  Quest,  there  is  no  internal  concept  of  icon,  and  these  icons  are  completely 
implemented  using  QAL  and  stored  in  the  shape  library,  which  allows  for  easy 
modification.  Since  the  shapes  in  a  frame  are  dynamically  linked  to  the  library,  changes 
made  to  the  library  shape  immediately  change  the  shape  in  the  frame.  This  feature  is  very 
useful,  especially  with  respect  to  icons.  The  color,  size,  and  appearance  of  the  icons 
need  only  be  changed  in  one  place.  Following  is  a  description  of  the  implementation  of 
one  of  the  icons,  the  Quit  icon.  When  the  student  selects  this  icon,  the  Quest  answer 
variable  is  set  to  the  string  "quit*".  Then,  the  answer  analysis  performs  an  Unconditional 
branch  to  the  frame  'ooga\  a  non-  existent  frame.  Branching  to  a  non-existent  frame 
causes  Quest  to  exit  the  course  and  save  where  the  student  is  in  the  lesson.  The  next 
time  the  student  enters  the  course,  Quest  will  restore  the  student  to  the  nearest 
bookmarked  page.  Quest  recommends  not  bookmarking  all  pages,  since  this  slows 
down  the  system,  but  it  has  been  our  experience  that  this  has  a  negligible  effect.  In 
addition  Quest  saves  the  values  of  all  embedded  statement  variables.  This  is  a  very 
useful  feature.  For  example,  the  history  stack  will  be  restored  to  the  same  state  it  was 
when  the  student  exited.  Each  of  the  following  icons  is  implemented  in  the  same  fashion. 
When  the  particular  icon  is  selected,  the  answer  variable  is  set. 


Conclusion 

Designing  good  courseware  is  difficult,  requiring  the  author  to  invest  a  large  amount  of 
time  and  effort  on  the  project.  On  the  basis  of  our  experience  using  Quest,  we  believe 
that  the  requirements  of  the  project  demand  more  than  a  single  designer.  Although 
authoring  systems  attempt  to  reduce  the  number  of  experts  necessary  to  create 
courseware,  we  are  convinced  that  this  is  not  feasible.  Authoring  systems  can  allow 
authors  who  are  not  computer  literate  to  construct  courseware,  but  there  are  many  other 
areas  that  must  be  addressed.  The  use  of  colors,  graphics,  and  animation  all  require 
some  deal  of  expe.lise  in  these  areas.  An  authoring  system  may  provide  the  tools 
necessary  to  create  quality  graphics  and  animation,  but  the  author  still  must  have  some 
knowledge  of  their  proper  use.  In  fact,  these  tools  may  make  it  easier  to  create  ineffective 
or  even  counter-productive,  rather  then  beneficial,  courseware  [Mau92]. 

Our  project  shows  that  the  ability  to  use  QAL  in  concert  with  Quest  is  essential. 
For  example,  using  QAL,  we  simulate  pop-up  windows  and  hype^ext.  The  power  of  a 
programming  language  is  almost  limitless,  even  a  rather  poor  language  like  QAL  The 
problem  with  using  QAL  is  that  it  essentially  reduces  Quest  to  an  authoring  language. 
Quest  can  not  be  considered  a  totally  complete  authoring  system,  since  some  important 
tools  can  only  be  implemented  using  QAL.  Despite  this,  Quest  can  still  be  used  by  non- 
programmers  to  create  some  useful  courseware,  especially  if  input  is  limited  to  the 
keyboard. 

Although  we  attempt  to  provide  many  of  the  features  of  a  complete  package,  there 
are  some  items  that  could  be  implemented  in  future  wori^.  There  needs  to  be  a 
considerable  increase  in  the  interaction  of  the  courseware.  The  feasibility  of 
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communicating  with  LEX  and  YACC  from  within  Quest  should  also  by  studied.  This 
communication  would  have  to  be  implemented  using  files  and  QAL 
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Abstract 

In  an  era  of  high  cost  technological  equipment  and  educational  budgetary  belt-tightening,  many  institutions  are 
finding  the  cost  of  keeping  pace  with  current  technology  prohibitive.  Many  institutions  find  it  necessary  to  make  due 
with  existing,  often  outdated,  equipment  for  use  in  their  existing  labs  as  well  as  in  the  development  of  new 
multimedia  labs.  This  paper  discusses  the  development  of  a  multimedia  lab  without  exceeding  the  constraints 
imposed  by  limited  funds.  The  discussion  includes  construction  of  the  tab,  the  challenges  encountered,  and  a  brief 
description  of  current  research  and  development  efforts. 


Introduction 

In  an  era  of  high  cost  technological  equipment  and  educational  budgetary  belt-tightening,  many  instiMions  are 
finding  the  cost  of  keeping  pace  with  current  technology  prohibitive.  Many  insti'tutions  find  it  necessary  to  make  due 
with  existing,  often  outdated,  equipment  for  use  in  their  existing  labs  as  well  as  in  the  development  of  new 
multimedia  labs.  At  the  University  of  Northern  Colorado  the  approval  of  a  new  doctoral  program  in  Educational 
Technology  made  it  necessary*  to  establish  a  multimedia  development  laboratory.  To  meet  this  need,  the 
Educational  Technology  faculty  proposed  a  multi-year  plan  which  would  provide  the  hardware  and  software  needed 
to  bring  the  technical  part  of  the  program  to  a  level  roughly  equivalent  with  many  corporate  development 
operations.  While  the  cost  of  the  proposal  greatiy  exceeded  the  available  funds,  the  need  for  a  multimedia 
development  laboratory  to  support  the  new  doctoral  program  remained.  This  paper  discusses  the  development  the 
multimedia  laboratory  without  exceeding  the  consti'aints  imposed  by  limited  funds.  The  discussi^^n  will  include 
construction  of  the  lab,  the  challenges  encountered,  and  a  brief  description  of  current  research  and  development 
efforts. 

While  on  the  surface  it  seems  that  the  function  of  the  multimedia  laboratory  is  to  support  the  courses  in  the 
Educational  Technology  program,  the  limited  resources  at  the  university  require  that  the  function  of  the  laboratory 
be  defined  in  more  concrete  terms  which  can  be  defended  in  budget  meetings.  Therefore  our  laboratory  is 
designed  to  serve  four  primary  purposes,  to  be  phased  in  over  the  next  several  years: 

•  interactive  technology  research 

•  development  of  marketable  products 

•  ti'aining  for  graduate  students,  faculty,  and  staff  in  the  use  of  interactive  technology,  and 

•  technology  testing. 
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Research 


Interactive  technology  research  requires  the  ability  to  run  existing  courseware,  develop  courseware,  and  conduct 
tests  of  the  courseware.  Further,  it  requires  a  facility  having  enough  in  common  with  other  facilities  on  campus  to 
allow  collaboration  with  programs  from  across  the  campus.  This  collaboration  is  necessary  to  provide  content  (and 
context)  for  study,  subjects  for  study,  and  an  avenue  to  begin  the  transformation  of  the  educational  process  across 
the  campus  to  one  which  maximizes  appropriate  use  of  technology,  which  can  only  be  done  through  a  focused 
research  effor  t  {The  effort  to  transfonn  the  educational  process  is  an  underlying  goal  of  the  institution,  and  was 
part  of  the  justification  for  beginning  the  development  of  the  laboratory.) 

The  use  of  the  facility  for  research  v;as  the  beginning  point  for  addressing  and  defining  the  needs  within  the 
laboratory  facility.  For  example,  the  need  for  some  commonality  with  facilities  across  campus,  as  v;ell  as  the  need 
to  provide  some  leadership,  defined  one  of  the  dimensions  considered  in  the  design  of  the  laboratory.  The  need  to 
run  existings  courseware  and  the  need  to  be  able  to  develop  our  own  courseware  defined  another  dimension. 
(Developnr;ent  requires  more  capable  hardware  and  software  than  does  running  existing  courseware,  in  general.) 

Development 

D^^velopinent  of  courseware  for  research  purposes  is  a  starting  point  However,  facilities  such  as  the  multimedia 
development  lab  require  more  funding  than  available  through  normal  budgetary  processes.  Since  the  lab  is  not  a 
general  access  student  lab,  it  is  not  eligible  for  funding  through  student  fees.  Yet,  the  demand  for  constant  upgrade 
must  be  met  in  order  to  remain  a  viaWe  resource  for  the  program  or  institution.  Therefore,  alternative  funding  must 
be  sought.  Building  on  the  research  base,  the  goal  of  commercially  viable  products  is  one  method  that  can  be  used 
to  obtain  some  revenue  specifically  for  the  laboratop/.  While  this  is  one  purpose  of  the  laboratory,  it  will  not  be 
implemented  during  the  first  several  years.  There  are  a  couple  of  major  reasons  for  the  delay.  First,  there  are  not 
enough  people  (students,  staff,  faculty)  with  the  required  expertise  to  produce  commercially  viable  products  at  this 
time.  As  the  laboratory  is  increasingly  used  in  support  of  research  and  coursework,  the  necessary  expertise  will  be 
devefcped.  The  second  reason  is  that  the  financial  details  and  intellectual  property  right  details  have  not  yet  been 
clearly  defined. 

Training 

In  order  to  meet  the  goal  of  transforming  educational  practice,  or  even  to  be  able  to  develop  and  use  courseware, 
peophJ  must  be  trained.  The  training  function  of  the  multimedia  lab  falls  into  three  categories.  Training  of  students 
and  fa.culty  active  in  the  development  and  research  efforts  centered  in  the  lab  was  begun  concurrent  with  the 
planning  and  implementation  of  the  lab.  it  began  with  a  small  group  of  students  and  one  faculty  member  and  was 
conducted  within  the  context  of  doctoral  level  courses  dealing  witii  the  development  of  courseware.  This  approach 
accomplished  the  goal  of  producing  a  core  group  of  trained  individuals  who  can  conduct  the  next  level  of  training 
withoit  th.e  additional  expense  of  outside  training. 

The  tv^o  remaining  categories  of  ti-aining  are  training  students  to  use  the  technology  to  support  their  classes  and 
training  faculty  to  develop  course  materials  and  use  the  technology  to  support  class  presentation.  Student  training 
began  with  a  pilot  study  during  the  Summer  1992  semester.  Training  for  faculty  began  with  the  Fall  1992  semester 
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Testing 

Once  a  critical  mass  of  both  students  and  faculty  have  been  trained  on  the  existing  technologies  and  have  begun  to 
effectively  implement  the  laboratory,  additional  technologies  will  be  evaluated  for  possible  inclusion  into  the  existing 
system.  This  testing  of  the  nevs/  equipment  and  technological  advances  for  compatibility  and  effecUveness  are  a 
goal  for  the  near  future 


Hardware  and  Software 

The  design  of  the  lab  required  consideration  of  hardware  and  software  already  in  place  both  for  research  and 
instruction.  From  that  base,  consideration  of  the  direction  of  cun'ent  and  future  research  and  development  efforts, 
as  well  as  the  course  requirements  for  the  new  Ph.D.  lead  to  the  specification  and  development  of  a  multi-platfomi 
(two  at  this  time)  laboratory  with  a  balance  of  software  tools  including  high  level  programming  languages,  software 
development  environments,  authoring  systems,  graphics  and  image  processing  tools,  and  desktop  publishing  tools. 

With  the  limitations  of  a  small  budget,  it  was  decided  that  the  initial  hardware  emphasis  would  be  MS-DOS  and 
Windows  based  hardware.  While  more  multimedia  software  development  tools  are  available  for  the  Macintosh 
platform,  the  Intel  based  hardware  provides  more  raw  processing  power  for  the  dollar.  We  were  able  to  purchase 
three  additional  486  based  computers  and  equip  them  with  a  minimal  set  of  software  tools  within  our  budget.  The 
486  computers  were  produced  by  a  local  company  which  had  won  the  low  bid  for  computing  equipment  for  the 
university.  While  the  computers  were  very  inexpensive,  there  were  some  compatibility  problems.  The  new 
computers,  and  to  some  extent  the  existing  hardware,  were  modified  by  the  faculty  to  include  third  party  (and 
inexpensive)  multimedia  components,  including  audio  boards,  CD-ROM  drives,  and  graphics  overlay  video  boards. 
In  making  the  purchases,  we  did  not  ignore  the  Macintosh  platform.  We  added  a  new  Macintosh  lisi  to  the 
laboratory,  allowing  the  use  of  existing  Mac  specific  peripherals. 

The  software  tools  provided  for  both  platforms  included  software  development  environments  (HyperCard, 
ToolBook,  Borland  C++),  authoring  systems  (eariy  version  of  AuthorWare,  Challenger),  graphics  tools  (Draw  and 
Paint  software),  desktop  publishing  (PageMaker),  and  presentation  software  (PowerPoint).  All  products  were 
available  with  substanSal  educational  discounts.  While  Oie  available  software  and  hardware  tools  are  limited,  they 
do  allow  the  construction  of  some  very  sophisticated  products  and  can  be  used  for  both  research  and  production  of 
commercial  grade  materials.  With  this  balance  of  resources,  the  lab  accommodates  the  widest  pos?ible  mission. 

Existing  equipment  included  one  '286  and  one  '386  pc  with  audio/video  modification,  one  Macintosh  lIsi,  1  Pioneer 
6000  laser  disc  player,  2  gray  scale  scanners  for  Macintosh,  1  gray  scale  scanner  for  PC,  three  Hewlett-Packard 
re-write  optical  drives,  one  LaserJet  II  printer  and  three  DeskvTiter  printers.  New  equipment  included  two  IBM 
compatible  '486  computers  which  were  upgraded  to  multimedia  personal  computers  (MPC's),  a  Pioneer  8000  laser 
disc  player,  an  additional  audio  amplifier,  a  color  scanner,  an  NEC  PC-VCR,  a  color  scanner,  and  a  '486  IBM 
compatible  to  be  used  for  the  File  Server. 

At  this  time,  we  have  a  reasonably  equipped  and  functional  multimedia  development  laboratory  capable  of 
supporting  the  research  efforts  of  faculty  and  students.  It  is  also  capable  of  supporting  the  training  purpose  within 
courses  and  for  faculty  on  the  small  scale  permitted  by  five  work  stations.  During  the  next  year,  several  additional 
wori^stations  and  more  software  will  be  added,  improving  both  the  capabilities  and  balance  of  platforms  available  in 
the  laboratory.  The  additional  stations  will  allow  fully  implementing  the  training  purpose  and  the  beginning  of  the 
materials  development  purpose. 
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The  Network 

Because  of  the  diverse  user  base,  Novell  version  3.11  was  chosen  as  the  network  operating  system.  Four  levels  of 
users  were  defined  using  Novell's  file  and  directory  security  -  Administrators  (full  access), 
Researchers/Developers  (read/write  access  to  group  directories),  Trainers  (read  only  rights)  and  Students  (read 
only  rights).  The  Research/Development  group  includes  faculty  and  doctoral  students;  the  Trainers  includes 
anyone  evaluating  or  instructing  with  training  materials,  primarily  masters  students  and  certain  undergraduates;  and 
the  students  receiving  training  are  primarily  undergraduates. 

Shared  access  to  certain  directories  on  the  server  are  granted  to  the  Developers  so  that  work  can  be  shared  and 
critiqued  by  peer  developers.  Students  are  given  somewhat  more  limited  access,  while  General  Users  are 
restricted  to  the  execution  of  software  only.  One  exception  to  the  above  is  that  any  user  granted  a  personal 
account  is  given  his  own  "home"  directory  for  personal  storage. 

Other  reasons  which  contributed  to  the  choice  of  Novell  included:  A  copy  of  version  2.1 5  was  available  for 
upgrade,  which  saved  on  the  cost  Novell  is  the  de-facto  standard  on  the  UNO  campus  with  1 1  Novell  servers 
supporting  various  labs  and  offices;  third-party  support  is  more  readily  available  for  Novell,  including  specific 
software  for  CD  drive  sharing;  and  more  brands  of  hardware  are  supported  by  Novell,  enabling  the  project  to  be 
assembled  from  lower  cost  components. 

The  diverse  user  group  also  led  to  the  use  of  twisted-pair  ethernet  topology.  Because  of  the  general  use,  high 
traffic  operating  environment,  cabling  can  be  damaged  more  easily,  and  twisted  pair  isolates  errors  to  single 
workstations  rather  than  "downing"  the  entire  network.  Twisted  pair  also  caanot  be  dis-connected  incorrectly  when 
equipment  is  moved  within  the  lab  or  to  a  classroom.  Ethernet  was  chosen  over  token-ring  because  it  performs 
well  in  a  small  networic  situation  and  ethernet  adapters  are  readily  available  at  lov/er  cost. 

The  sharing  of  devices  and  directories  contributes  to  the  co-operative  feeling  the  lab  hopes  to  promote,  and  the 
total  cost  of  the  lab  was  reduced  by  networking.  Peripheral  devices,  including  CD  drives  and  printers,  are  shared 
by  all  wori^stations  and  do  not  need  to  be  duplicated  at  each  computer.  Sharing  of  CD  was  accomplished  using 
"Map  Assist"  by  Fresh  Technology.  Novell's  own  print  services  were  used  for  printer  sharing. 


Pitfalls 

The  development  of  a  multimedia  lab  intended  to  serve  multiple  levels  of  users  is  always  accompanied  by 
problems.  Some  of  the  problems  we  encountered  were  the  result  of  serving  multiple  levels,  some  were  the  result  of 
implementing  the  laboratory  within  the  constraint  of  ttie  limited  budget.  For  example,  an  (apparently)  inexperienced 
user  erased  several  files  on  one  woricstatioi ..  in  the  eariy  stages  of  implementation,  it  was  not  uncommon  to  find 
that  someone  had  "cleaned"  the  lab  and  misplaced  or  discarded  important  manuals  and  original  disks.  (At  one  time 
both  copies  of  the  booklets  providing  technical  information  for  the  network  cards  were  lost.)  In  addition,  students 
and  faculty  using  otiier  equipment  located  in  the  same  area  as  the  lab  frequentiy  moved  or  disconnected  the 
cabling.  The  disconnects  were  most  likely  accidental,  since  the  cabling  was  frequentiy  exposed,  though  outside  of 
the  normal  traffic  pattern.. 

Equipment  failures  were  also  problematic  in  creating  the  lab.  A  faulty  hard  drive  controller  in  the  server  was 
replaced  by  the  computer  retailer  with  a  different  brand  not  recognized  by  the  Novell  software.  A  delay  of  several 
weeks  was  experienced  in  locating  Novel!  drivers  for  the  new  controller.  (These  drivers  are  normally  supplied  with 
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the  controller  card,  but  the  retailer  did  not  forward  them  to  us.)  Once  the  server  was  working,  it  was  discovered  that 
the  network  cards  supplied  were  not  compatible  with  the  *486  computers,  even  though  they  were  supplied  by  the 
same  retailer.  It  was  determined  after  several  hours  of  phone  troubleshooting  with  the  network  card  manufacturer 
that  the  cards  were  unable  to  ''keep  up"  with  the  '486  bus  speed.  Another  delay  of  several  weeks  was  experienced 
while  troubleshooting  and  testing  other  network  interface  cards.  Finally,  Intel  brand  cards  were  selected,  allowing 
the  network  to  operate  normally. 

Equipment  failures  and  people  problems  merged  in  several  cases  as  the  i«o  was  created.  Due  to  the  limited 
funding,  no  tape  or  optical  back-up  hardware  was  available  and  system  back-ups  were  limited  to  floppy-based 
back-ups  of  critical  system  setups  and  files.  While  this  did  afford  some  protection,  it  did  not  prevent  many  hours 
spent  reconstructing  systems  following  a  system  crash  due  to  human  errors. 


Training 

Training  is  a  key  component  of  the  implementation.  Faculty  and  graduate  student  users  include  researchers, 
developers,  evaluators,  and  end  users.  Different  levels  of  training  are  required  for  each  type  of  user,  since  each 
type  requires  a  different  level  of  access  to  resources.  For  those  more  familiar  with  another  operating  environment 
training  is  being  developed  to  facilitate  the  transfer  to  Windows  or  Macintosh  based  systems.  However,  only  limited 
training  more  in  the  form  of  apprenticeship  is  being  conducted  at  this  time. 

Although  there  is  an  insufficient  number  of  personnel  with  the  prerequisite  skills  to  implement  commercially  viable 
products  at  this  time,  there  is  a  program  in  place  to  change  this.  Most  of  the  training  has  occurred  for  doctoral 
students  and  in  class.  Classes  have  included  the  use  of  authoring  and  programming  languages  as  well  as  the 
implementation  and  management  of  interactive  video  and  instructional  systems. 

An  important  component  of  the  training  is  the  use  of  graduate  students  in  training  others.  Two  graduate  assistants 
and  two  graduate  practicum  students  were  assigned  to  learn  the  netjvork  and  basic  operations  of  the  multimedia 
tools.  Their  training  is  being  provided  in  part  by  the  university  computer  center  (on  the  network)  and  by  faculty  (on 
the  multimedia  tools).  The  remainder  is  self-teaching.  These  students  are  in  the  process  of  creating  job  aids  for 
others  involved  in  the  lab.  These  aids  will  eventually  include  the  multiple  levels  of  instruction. 


Current  projects 


Currently  projects  are  being  developed  within  classes,  for  dissertations,  and  for  the  Taliesin  Project,  a  large 
research  and  development  project  (Smith  &  Westlioff,  1992).  All  are  real  world  applications  that  are  expected  to 
outlive  the  academic  assignment  that  launched  them.  In  addition  to  class  time,  faculty  have  worked  with  students 
outside  of  class  to  support  several  research  projects.  Many  of  these  projects  were  initiated  within  courses  and 
continued  the  following  semester.  Several  of  these  are  described  in  the  following  paragraphs. 

Munson  and  Smith  (1991)  developed  a  hypermedia  model  that  investigated  two  different  navigational  schemes, 
icons  and  concept  maps.  Although  this  series  of  studies  was  initiated  before  the  implementation  of  the  lab,  they 
were  of  major  concern  in  the  planning  and  implementation  of  the  lab.  In  addition,  most  of  the  actual  studies 
occurred  in  the  development  lab.  They  tested  these  within  a  physics  content  with  both  education  and  physics 
students.  Lightner,  Smith,  and  Reed  (1992)  developed  a  hypertext  program  modeled  after  two  different 
instructional  theories  that  investigated  differences  in  achievement  by  accounting  students.  Munson,  Smith,  and 
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Janson  (1992)  have  developed  a  prototype  of  a  model  allowing  students  to  explore  animal  symbolism  and  then 
create  their  om  stories  using  a  videodisc  as  an  information  resource.  This  project  called  Wildlife  Wonders 
examines  the  implementation  of  one  portion  of  the  Taliesin  Project.  They  have  submitted  proposals  to  fund  further 
development  of  this  project. 

Two  additional  projects  were  developed  in  conjunction  with  the  development  lab.  The  first  project  was  developed 
for  use  in  middle  school  science  classrooms  and  provides  students  the  medium  to  create  and  manipulate  a 
database  for  classifications  of  animals.  The  student  classification  schemes  used  to  organize  the  database  were 
investigated  in  relation  to  the  selection  of  the  phylum,  kingdom,  class,  habitat,  and  geographic  location.  The 
second  project  involved  exploring  the  preconceptions  and  mental  efforts  of  students  when  using  computer  assisted 
interactive  video  and  instructional  television. 

An  additional  class  project  has  evolved  into  a  research  project  in  which  graduate  students  were  used  to  train 
undergraduate  students.  This  project  was  designed  and  developed  to  address  the  issue  of  preservice  teacher 
technology  training  and  to  demonstrated  the  use  of  multimedia  (hypermedia),  desktop  publishing,  and  analog  video 
technologies  within  the  educational  environment.  The  research  project  teamed  graduate  students  with 
undergraduate  preservice  teachers  education  majors  to  complete  a  project  using  one  of  the  three  technologies, 
with  the  graduate  students  responsible  for  the  training. 

An  important  facet  of  the  research  and  development  projects  housed  in  the  development  lab  is  collaboration 
between  programs.  The  business  school  has  been  involved  in  one  project  and  several  are  currently  at  the 
inception  stage.  The  major  collaborating  program  has  been  the  Kinesiology  and  Physical  Education  department. 
Three  major  videodisc  projects  have  been  locally  planned,  developed,  and  produced.  After  the  discs  were 
pressed,  Educational  Technology  graduate  students  developed  the  softv/are  for  the  discs. 

The  first  two  projects  were  produced  to  train  PE  teachers  to  analyze  motor  skills  in  swimming  (Mathias,  1990- 
developed  prior  to  the  lab,  but  used  each  semester  within  the  lab)  and  tennis  (Chung,  Munson,  Smith,  &  Phillips, 
1 992).  The  PE  department  is  hopeful  that  these  discs  and  others  can  be  used  outside  of  class  to  help  them  reduce 
the  number  of  credits  required  by  their  students  and  increase  quality'  at  the  same  time.  In  a  second  phase,  the 
tennis  program  is  now  being  used  with  non-PE  students  to  investigate  the  program*s  impact  on  persona!  tennis 
performance.  A  third  project  filmed  in  a  basketball  class  was  used  with  the  physical  education  teacher  assessment 
instrument  to  investigate  the  performance  of  the  teacher  in  a  classroom  and  identify  teacher  behaviors  (Norton, 
1992). 


Recommendations 

It  is  clear  that  the  limited  resources  of  the  multimedia  development  laboratory  are  being  used  effectively.  We 
expect  the  use  to  increase  rapidly  and  begin  to  include  desktop  publishing  and  video  production  efforts  as 
hardware  and  time  are  available.  Indeed,  we  expect  demand  for  the  facility  to  shortly  exceed  availability.  Despite 
the  success,  there  are  several  recommendations  for  other  institutions  considering  the  installation  of  a  muiti-user 
development  lab,  based  on  our  experience. 

Cabling  should  be  planned  and  routed  carefully,  even  if  fault-tolerant  cabling  like  unshielded  twisted-pair  is  used. 
Name-brand  network  cards  will  save  a  great  deal  of  trouble  shooting  of  the  Ian  and  cost  only  a  few  dollars  more 
than  "generic"  cards.  Name  brand  PC's  should  also  be  considered  if  not  cost  prohibitive,  and  name-brand,  certified 
parts  are  a  must  (e.g.  Adaptec  SCSI  controllers,  etc).  Backup  equipment  and  fault  tolerance  systems  should  be 
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budgeted  for  as  well  (e.g.  Uninterruptable  Power  Supply  for  the  file  sen/er,  tape  backup  unit(s),  master  software 
files  under  lock  and  key). 

Graduate  assistantship  or  other  on-site  iab  administrator  is  recommended  to  assist  with  and  oversee  the  lab, 
maintain  inventory,  and  trouble-shoot  problems  as  they  arise.  The  location  of  the  lab  should  be  isolated  from  high- 
traffic  flow  where  possible.  And,  access  to  the  lab  should  be  restricted  to  people  already  trained  and  working  on 
development  projects  or  people  undergoing  training.  The  lab  should  not  be  opened  to  casual  use  or  general 
course  support  With  limited  resources  and  high  demand,  research  and  development  efforts  must  take  priority. 
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The  Use  of  Computer-Based  Instractional  Simulations  for  Teaching  Hiotography  Skills 

Charles  F.  Belinski,  Vie  University  of  Georgia 

With  advances  in  authoring  systems  and  the  ability  to  display  photographic  quality  images  on 
computers,  computer-based  instructional  simulations  can  be  created  and  implemented  for  a  wider 
variety  of  instructional  topics.  This  session  will  discuss  the  design,  development,  and 
implementation  of  a  series  of  computer-based  instnictional  simulations  that  were  developed  to 
assist  in  the  delivery  and  instruction  of  basic  photography  topics. 

Instructional  simulations  can  be  separated  into  two  major  categories.  Simulations  can  either 
teach  about  a  topic  (informational  simulations)  or  they  can  instruct  how  to  perform  an  action  or 
series  of  actions  (operational  simulations).  The  design  and  development  of  informational  and 
openitional  simulations  create  special  concerns.  The  amount  of  realism  presented  in  instructional 
simulations  directly  influences  the  effectiveness  of  the  simulations.  Issues  such  as  the  application 
of  various  levels  of  visual  and  functional  realism  in  relation  to  user  experience  and  other  topics 
need  to  be  addressed  when  designing  and  developing  an  instructional  simulation. 

Demonstrations  will  be  done  of  representative  samples  of  both  informational  and  operational 
simulations.  Recommendations  for  the  design,  development,  and  implementation  of  computer- 
based  instructional  simulations  will  be  discussed. 


Designing  Computer-based  Assessment 

James  H.  Cowardin,  Precision  Learning  System 

The  field  of  automated  training,  if  not  the  entire  field  of  training,  begs  for  a  rigorous  assessment 
measure  to  augment  of  replace  the  ubiquitous  but  weak  score  of  percent  correct.  The 
straightforward  answer  is  to  adopt  the  concept  of  fluency,  rate  of  accurate  responding.  This 
measure  certainly  applies  wherever  percent  correct  is  appropriately  used  now,  and  it  probably 
has  even  more  potential.  Rate  of  correct  responding  provides  much  more  information  than  the 
correct/total  ratio.  Our  experience  shows  that  learners  find  fluency  measures  interesting  enough 
to  motivate  them  to  persist  in  learning  in  as  percent  correct  does  not.  Developing  training  or 
assessment  based  on  fluency  requires  new  thinking  about  instructional  design.  But  the  technique 
applies  across  media  and  strengthens  CBT  limited  to  ASCII  text  as  will  as  video-disk  interactive. 

Fluency  measures  anchor  a  new  and  unique  learning  model.  Generative  Instruction,  which  states 
that  higher  level  cognitive  performance,  such  as  problem  solving,  depends  on  many  more  basic 
level  skills  that  are  learned  to  fluency  and  called  upon  as  needed,  or  recombined  to  meet  the 
present^^  need.  This  technique  adds  emphasis  on  basic  skills  to  the  current  tendency  to 
presevt  ie  on  (imperfect)  simulations  of  high  level  performance.  Assess^  lent  keys  Generative 
InstruCv4jn  in  providing  navigational  aids  by  which  instructors  can  pilot  safe  passage  for  the 
learner.  Fluency  adds  precision. 
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Are  Softwai-e  Quality  Factors  a  Relevant  Measure  of  CBT  Quality? 

Michael  J.  Hillelsohn,  McDonnell  Douglas 

There  is  more  to  high  quality  CBT  than  just  instiiictional  effectiveness.  If  we  treat  CBT  as  a 
software  application  can  we  capitalize  on  existing  software  tools  to  describe  the  quality  of  our 
courseware?  Quality  does  not  have  to  be  an  amorphous  concept,  best  described  as  "I  know  good 
quality  when  I  see  it!"  Rome  Air  Development  Center  (R.A.D.C.)  has  been  researching 
software  quality  factors  for  many  years.  The  thirteen  quality  factors  which  are  the  basis  of  their 
approach,  provide  a  consistent  lexicon  for  communicating  software  quality  requirements  to 
developers,  and  then  measuring  the  quality  characteristics  of  the  final  product.  Each  quality 
factor  has  attributes  and  metrics  associated  with  it,  so  that  the  quality  of  the  application  can  be 
measured.  *  In  this  session  the  quality  factors  and  their  attributes  will  be  presented,  a 
demonstration  of  a  decision  aid  to  prioritize  the  factors  will  be  demonstrated  and  we'll  discuss 
the  applicability  of  such  tools  to  assessing  the  quality  of  CBT  programs. 


Effective  Screen  Design  for  Computer-Ba^^ed  Training  and  Interactive  Video 

William  D.  Milheim,  Perm  State  Great  Valley;  Brenda  Bannan  Haag,  Penn  State  University 

One  area  of  particular  importance  for  the  development  of  computer-based  training  and  interactive 
video  programs  is  the  issue  of  screen  design,  which  can  have  a  very  significant  impact  on  both 
the  overall  effectiveness  and  the  learner  appeal  of  a  particular  instructional  project.  The  overall 
field  of  screen  design  encompasses  a  number  of  sub-topics  including  items  such  as  color, 
balance,  use  of  graphics  and  text,  windowing,  object  location,  button  usage,  and  general  learner 
control  features,  among  others.  Each  of  these  items  is  important  to  the  instructional  design 
process  for  the  production  of  an  easy-to-use  program  that  is  effective,  yet  provides  a  comfortable 
learning  environment. 

Based  on  review  of  pertinent  literature  and  the  practical  experience  of  the  presenters,  a  number 
of  screen  design  guidelines  have  been  developed.  These  include: 

-limiting  screen  information  to  relevant  detail, 

-using  a  limited  amount  of  text  on  each  screen, 

-using  standard,  relatively  large  fonts  for  text-based  information » 

-using  graphics  instead  of  text  when  possible, 

-using  color  where  appropriate,  with  a  limited  number  of  colors  per  screen, 
-using  windows,  tables,  or  other  forms  of  partitioning  to  structure  information  on  each 
screen,  and 

-separating  instructional  information  from  navigational  aids  in  a  systematic  way. 
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Quality  CBT  -  The  Quick  Way 

Jeff  Oliver,  Castle  Learning  Systems  Ltd. 


The  conventional  theoretical  mode]  for  courseware  development  has  significant  shortcomings 
when  writing  CBT  for  commercial  clients.  A  practical  methodology  needs  to  overcome  the 
limitations  of  paper-based  designs,  to  consider  implementation  issues  early  in  the  design  process, 
to  keep  the  client  happy  and  work  within  budget. 

"The  Practioners*  Method"  is  an  evolving,  working  methodology  which  has  been  applied 
successfully  in  a  number  of  projects.  The  stages  of  the  process  involve: 

-Minimising  less  productive  parts,  such  as  finely  defined  objectives. 

-Extracting  subject  matter  and  producing  narratives,  not  detailed  layout  forms. 

-Creating  a  courseware  prototype. 

-Reusing  well-tested  elements  to  minimize  bugs. 

-Costing  the  script  and  adjusting  the  design  to  fit  the  budget. 

-Importing  text  from  word  processed  scripts  directly. 

-Producing  intermediate  versions  of  the  courseware  for  review. 

-Systematic  acceptance  testing. 

The  method  challenges  the  notion  that  the  programming  part  of  CBT  should  be  a  mechanized 
inputting  process  but  rather  that  the  programmers  should  interpret  the  design  script  in  a  creative 
way.  This  presentation  discusses  the  method  in  detail,  with  illustrations  drawn  from  past 
projects. 

Credentialing  Courseware  Developers:  A  Postgraduate  Program 
in  Computer  Based  Learning 

Rod  Sims,  University  of  Technology,  Sydney  Australia 

In  1992,  the  University  of  Te^''  lOlogy  Sydney  (UTS)  introduced  a  Graduate  Diploma  in 
Computer  Based  Learning  (GDCBL).  The  program  runs  for  two  years  part-time  (one  day  per 
week  per  semester)  or  for  three  block  (residential)  weekends  per  semester.  The  program  was 
developed  as  a  result  of  changes  in  students  requirements  and  legislative  changes  (Training 
Guarantee  Act)  for  Australian  business.  In  addition,  there  is  now  an  industry-wide  push  for 
Competency  Based  Training  (CBT),  which  has  resulted  in  an  emphasis  on  multi-skilling  and 
award  restructuring.  Within  this  environment,  the  demand  for  training,  and  training  alternatives, 
has  increased  -  and  programs  such  as  the  GDCBL  are  in  demand.  The  value  of  the  program  can 
be  emphasized  in  terms  of  four  major  factors: 

Course  Flow,  designed  to  present  individual  subjects  in  a  sequence  which  reflex^ts  the 

development  methodologies  for  interactive  training  materials. 
Delivery  Strategies,  providing  both  day  and  block  (or  lesidential)  attendance  patterns. 
Assessment  Strategies,  incorporating  techniques  to  integrate  assignments  into  cohesive 
products. 

Technology  Change,  such  that  new  technologies  are  integrated  into  the  program  as 
required. 


Meta-CBT  2:  The  FinaJ  Product 

Rod  Sims,  University  of  Technology,  Sydney  Australia 


The  endeavor  to  produce  effective  courseware  for  high-order  skills  remains  a  challenge  into  the 
1990*  s.  While  methodologies  of  Instructional  Systems  Design  define  the  content  structure,  the 
creative  ability  to  transform  subject  matter  into  performance-oriented  instiiictional  events  is 
elusive.  A  strategy  to  involve  the  learner  through  integration  with  the  content  is  proposed.  The 
development  of  a  meta-CBT  course  (CBT  about  CBT)  provided  the  opportunity  to  create  an 
environment  in  which  the  learner  actively  participates  in  the  coursev/are  development  process, 
compared  to  learning  about  those  processes.  This  strategy  provides  an  alternative  to  traditional 
stimulus-response  courseware  structures  and  sets  a  new  standard  for  learner  involvement  and 
interaction.  The  technique  may  also  be  viewed  as  creating  a  virtual  environment  in  which  the 
learner  can  explore  the  essential  relationships  of  the  focus  domain.  Of  equal  significance  is  the 
opportunity  to  implement  such  learner-integrated  interactions  using  traditional  frame-based 
authoring  tools,  demonstrating  the  essential  contributions  made  by  the  human  factor  in 
courseware  structures.  This  approach  is  now  being  applied  in  many  interactive  environments, 
and  provides  a  model  for  future  courseware  development. 


A  HyperCard/Hypermedia  Courseware  Package  Entitled: 
"Ticked  Off!  -  A  Resource  for  Tick  Identification" 

Wendy  Snetsinger,  Penn  State  University 

Computer-based  training  offers  wonderful  opportunities  for  engaging  the  learner  through  the  use 
of  animation,  unique  graphics,  appealing  screen  design,  and  other  techniques.  The  learner  can 
become  an  active  participant  through  the  courseware's  personalized  strategies  and  allowance  for 
user  control  over  options.  At  the  same  time  feedback  and  guidance  can  be  provided  based  on 
performance;  thus  CBT  is  ideal  for  independent  learning. 

A  hypercard/hypermedia  courseware  package  on  tick  identification  is  currently  being  developed 
by  the  Department  of  Entomology  at  Penn  State  University  in  cooperation  with  The  United 
States  Center  for  Disease  Control,  Ft.  Collins,  CO.  This  program  is  designed  to  address  a 
growing  health  problem  in  the  United  States  and  throughout  the  world  -  that  of  Lyme  and  other 
tick-borne  diseases.  Correct  tick  identification  is  crucial  for  the  accurate  diagnosis  of  these 
infections,  since  most  tick-borne  diseases  are  species  specific. 

Recognizing  the  broad  range  of  experience  and  interest  of  the  potential  users,  the  courseware 
employs  learning  theory  to  customize  needs  and  requirements  to  obtain  various  levels  of 
mastery.  Fach  unit  has  its  own  enabling  objective3,  and  the  lessons  provide  choices  of  how  the 
material  is  presented  -  through  conventional  strategies  or  through  humorous  metaphors, 
Microworld  simulations  are  presented  to  challenge  higher  level  thinking  and  problem  solving. 
Interactive  video  materials  and  references  are  incoiporated  to  provide  general  information  on 
related  material. 
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The  Ford  Experience 

Timothy  V/.  Spamaus  and  Laurie  Dienen  The  Emdicium  Group  Inc.; 
Jackie  Binkert,  Ford  Motor  Co, 

Ford  Motor  Company  has  committed  to  World  Class  Timing,  a  plan  to  accelerate  product 
development  as  a  way  of  improving  competitiveness  and  profitability.  Bringing  about  thai 
dramatic  change  requires  changes  to  the  organization  and  culture, 

A  cross-functional  team  derived  a  strategy  of  using  training  as  a  key  tool  to  bring  about 
organizational  change.  We  have  built  an  unusually  close  relationship  that  can  serve  as  a  model 
to  others  in  the  same  situation. 

Needs:  An  extended  needs  analysis,  including  focus  groups,  interviews,  reviews  of 
project  data  and  documentation,  determined  that,  in  addition  to  training,  needs  included 
cultural  changes  and  systems  approaches. 

Strategies:  The  approach  is  to  change  organizational  behaviors,  through  individual 
learning,  organizational  learning,  process  consulting,  and  a  commitment  to  continuous 
improvement. 

Processes:  We  built  a  series  of  interventions,  including  mass  communications,  training, 
organizational  learning,  team  consulting,  and  process  documentation.  Training  included 
classroom  and  hypermedia,  both  designed  to  integrate  product  development  processes. 

Results:  Preliminary  results  are  very  encouraging.  Individual  and  organizational 
acceptance  is  very  good.  Top  management  commitment  to  the  strategies  and  processes, 
better  definitions  of  product  development  processes,  and  successful  startups  of  product 
development  teams  point  toward  better  products. 

Scoping  a  Program  for  Delivering  CBT  to  your  Users:  A  Hands-On  Planning  Session 

Adele  Suchinsky,  US  General  Accounting  Office 

A  wealth  of  excellent  off  the  shelf  computer-based  training  (CBT)  and  interactive  videodisc 
(IVD)  is  available  to  business,  government  and  academia  for  use  in  a  self-paced  training 
environment.  Top  management  support,  however,  is  not  necessarily  the  only  element  required 
for  you  to  ensure  to  successful  implementation  of  a  multi-media  Learning  Center  or  a  distributed 
training  program. 

The  initial  challenge  is  to  identify  the  place  of  CBT  or  self-paced  training  within  the  context  of 
your  training  environment.  Once  the  need  for  the  program  is  established,  the  individual  tasked 
with  making  the  program  a  reality  is  faced  with  successfully  addressing  a  number  of  critical 
steps  in  the  implementation  process.  The  key  issues  include  budget,  staffing,  marketing, 
tracking,  evaluation,  facility,  courseware  selection,  determination  of  need/correlation  to  existing 
curriculum,  and  design  decisions  for  effective  implementation. 
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This  hands-on  planning  session  will  take  you  step  by  step  through  these  elements  Qn6  will  assist 
you  in  identifying  the  issues,  constraints,  and  alternatives  for  you  to  consider  for  implementing 
such  a  program  within  you  organization.  Through  completion  of  prepared  worksheets  and 
interactive  group  discussions,  you  will  identify  and  capture  thoughts  and  ideas  related  to 
implementing  a  self-paced  training  program. 
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Computer  Graphics: 
Implications  for  Instructional  Design 

ly 

lih^uan  Chanlin,  James  It  Okey,  &  David  Reinking 
University  of  Georgia 
Athens,  GA  30602 


Abstract: 


Computer  graphics  in  instructional  materials  support  learning  in  various 
ways.  They  aid  cognition  by  providing  stimuli,  retrieving  prerequisite  knowledge, 
encoding  concepts,  deciphering  information,  and  sustaining  motivation  and 
interest.  From  a  reviev/  of  the  literature,  guidance  for  using  graphics  related  to 
each  of  these  cognitive  processes  will  be  summarized  in  the  form  of  guidelines  for 
graphic  design.  Examples  illustrating  how  these  guidelines  can  be  employed 
with  computer-based  materials  will  be  presented. 

Introduction: 

Researchers  agree  that  the  use  of  multiple  representations  facilitates  the 
understanding  of  instructional  content  (Goldenberg,  1988).  The  need  for  multiple 
representations  is  that  different  learners  have  different  capabilities  for 
pi  ocessing,  understanding,  and  retaining  information,  and  no  single  mode  of 
presenting  information  works  equally  well  for  everyone  (Taylor  &  Cunniff,  1988). 
Describing  information  or  concepts  verbally  does  not  guarantee  the  learner's 
understanding  of  a  concept  or  a  fact.  Graphics  provide  an  additional 
representation  to  help  learners  comprehend  the  concepts  that  might  not  be 
understood  through  textual  description. 

For  increasing  motivation  to  learn,  gi'aphics  have  been  widely  used  as  an 
important  strategy  in  designing  instruction  for  various  contents  areas,  together 
with  various  delivery  systems.  Learning  is  facilitated  when  instruction  is 
appeaUng  to  learners  (Keller,  1983;  1987a;  1987b;  1988),  Including  graphics  in 
computer  instructional  materials  can  help  learners  develop  positive  attitudes 
toward  reading.  Graphics  make  learning  become  more  pleasant,  especially 
among  young  children  (Samuels,  1970).  Since  learning  requires  considerable 
mental  effort  in  relating,  reasoning,  and  reconstructing  information,  the  use  of 
graphics  may  motivate  learners  to  sustain  mental  effort  in  the  face  of  difficult 
learning  tasks.  Used  in  conjunction  with  verbal  materials,  graphics  also  provide 
a  more  concrete  representation  to  help  learners  conceptualize  the  information 
given.  Learning  becomes  more  interesting  when  comprehension  is  guided  by 
concrete  representations  (Keller,  1983).   Designing  instruction  that  appeals  to 
learners,  encourages  their  active  involvement  and  facilitates  persistence  in 
learning. 
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The  use  of  graphics  designed  to  complement  texts  can  significantly  improve 
student  achievement  of  certain  types  in  learning  task  (Hegarty,  Carpenter,  & 
Just,  1991).  However,  the  specific  type  of  graphics  that  wiU  be  most  effective 
depends  upon  the  student's  abilities  and  characteristics,  and  the  level  of 
information  processing  required  hj  a  learning  task  (Dwyer,  1988). 

The  purpose  in  this  paper  is  to  examine  how  graphics  can  assist  cognitive 
processing  as  viewed  from  the  perspective  of  enhancing  memory  and  motivation. 
The  cognitive  theories  that  support  the  use  of  graphics  in  memory  and  motivation 
are  discussed.  The  empirical  studies  addressing  the  effect  of  graphics  on  learning 
are  also  reviev/ed.  It  is  also  the  intent  in  this  paper  to  provide  prescriptions  and 
guidelines  for  designing  graphics  for  use  m  computer-assisted  instructional 
materials. 

Background  for  The  Use  of  Graphics: 

The  use  of  graphics  in  instruction  has  been  shown  to  support  the  following 
cognitive  purposes:  providing  stimuli,  retrieving  prerequisite  knowledge, 
encoding  concepts,  deciphering  information,  and  sustaining  motivation  and 
interest.  For  providing  stimuli,  graphics  are  used  for  obtaining,  sustaining,  and 
directing  a  learner's  attention  to  the  important  information  (e.g.  Levie  &  Lentz, 
1982;  Reynolds  &  Baker,  1987).  For  retrieval,  graphics  can  be  employed  for 
metaphoric  reasoning  or  for  relating  new  information  to  information  that 
students  are  already  familiar  with  so  that  new  learning  can  be  achieved  through 
a  learner's  own  reconstruction  (Kaufmann,  1985;  Reed,  1985).  For  decoding,  a 
picture  can  provide  a  concrete  referent  to  explain  an  abstract  concept  and  to  guide 
students  learning  through  visualizing  the  elements  and  the  relationships 
between  concepts  (Reid  &  Beveridge,  1986).  For  encoding,  pictures  can  help  people 
rehearse  to-be-remembered  materials  (Beck,  1987).  An  image  also  can  be 
constructed  by  people  through  pictures  or  imagery  instruction.  The  contrived 
image  extracts  the  key  concepts  given  from  verbal  information  so  that  the 
information  can  be  rehearsed  and  remembered  easily  (Alesandrini  &  Rigney, 
1981;  Morris  &  Hampton,  1983;  Pressley,  CarigUa-Bull,  &  Deane,1987).  Providing 
graphics  in  instruction  can  stimulate  emotional  arousal,  which  may  be  helpful  in 
encoding  information  (Levie,  1987).  For  sustaining  motivation,  graphics  may 
stimulate  learners'  interests  so  that  they  gain  enjoyment  and  generate  positive 
attitudes  toward  the  learning  materials  (Eisner,  1985). 

Knowing  how  to  use  graphics  efTectively  is  important  for  instructional 
designers  because  of  the  key  role  graphics  can  play  in  motivation  and  learning 
(Faiola  &  DeBloois,  1988;  Hannifin  &  Peck,  1988;  Marcus,  1992).  The  role  of 
instructional  materials  is  to  provide  the  external  learning  conditions  that  activate 
appropriate  internal  learning  processes  (Gagn6,  Briggs,  &  Wager,  1992).  Design 
techniques,  such  as  grid  systems,  typography,  color  palettes  and  animation,  or 
video  capture  capabilities,  allow  designers  and  developers  to  produce  vivid 
gi^aphics  by  using  advanced  technology.  However,  the  merits  of  employing 
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graphics  rests  mainly  on  their  capability  to  enhance  memory  and  comprehension 
during  learning.  The  role  of  a  graphic  designer  should  be  more  than  just  to 
producing  graphics  that  appeal  to  the  eye.  Moreover,  a  graphic  designer  should 
focus  on  applying  design  principles  and  research  findings  when  developing 
graphic  materials  (Faiola  &  DeBloois,  1988). 

'rhe  Psychological  Effect  of  Graphics 

Graphics  can  be  presented  in  forms  that  vary  from  simple  iconic  objects  to 
elaborate  and  realistic  pictures.  Graphics  can  be  concrete  or  abstract  and  may 
convey  analogical  or  representational  concepts  (Alexandrini,  1985).  Graphics  can 
also  present  information  inferentially.  For  example,  arrows  represent  directions, 
and  so  do  point-     fingers.  To  be  meaningful,  people  seeing  such  graphics  must 
understand  the  inferences  in  the  analogy.  How  well  graphics  benefit  certain 
groups  of  learners  depends  on  how  well  the  gi^aphics  relate  the  knowledge  and 
experience  that  learners  already  have  and  the  knowledge  and  expeiience  they  are 
expected  to  acquire. 

Meaningful  stimuli  are  by  definition  the  essence  of  communicating  intended 
concepts.   Although  it  is  assumed  that  the  more  stimuli  the  representation  can 
provide,  the  better  it  can  Tacilitate  the  process  of  learning  (Gibson,  1971),  providing 
stimuli  does  not  necessarily  guarantee  effective  response  unless  they  are 
meaningful  to  learners  (Franzwa,  1973).    Franzwa  reported  that  pictures 
classified  as  highly  meaningful  were  more  easily  learned  than  those  receiving  a 
lower  classification  in  meaningfulness.  The  surface  components  of  both  verbal 
and  pictorial  stimuli  should  be  relevant  and  coherent  with  the  underlying 
concepts  that  the  instruction  is  intended  to  provide  (DeBeaugrande  &  Dressier, 
1981).  Learning  from  graphical  information  is  not  simply  stimuli  di-iven;  it  also 
involves  relating  learners'  knowledge  and  the  new  concepts  to  be  learned  (Lindsay 
&  Norman,  1977).  Variations  among  learners  might  influence  the  process  of 
interpreting  graphic  materials, 

Sunnorting  Cognitive  Theories 

The  superiority  of  pictures  over  words  in  memory  has  been  studied  and 
addressed  in  several  cognitive  theories,  such  as  dual  coding  theory,  level  of 
processing  theory,  and  the  sensory-semantic  model.  The  dual  coding  theory 
(Paivio  1971,  1983,  1986,  1991)  suggests  that  pictures  and  words  activate 
independent  imaginal  and  verbal  codes.  However,  the  availability  of  this  dual 
code  differs;  that  is,  a  verbal  code  for  a  picture  is  more  available  than  an  imaginal 
code  for  a  word.  Pictures  are  better  than  words  because  they  are  more  easily  dual 
coded  (Paivio,  1983,  1986,  1991).  From  a  level  of  processing  view  (Craik  & 
Lockhart,  1972),  when  information  is  deeply  or  elaboratively  processed,  retention 
will  be  enhanced.  Pictures  used  in  instruction  might  induce  a  deeper  encoding 
because  a  concrete,  meaningful  representation  is  extracted.   Nelson  (1979) 
advocated  a  sensory-semantic  mode).  It  was  suggested  that  the  picture  code  is 
superior  to  verbal  The  superiority  effect  of  pictures  is  attributed  to  a  more 
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distinctive  sensory  code;  semantic  judgements  can  be  made  more  rapidly  with 
picture  chan  with  words.  Although  these  theories  have  different  explanations 
and  no  theory  provides  an  imassailable  viewpoint  concerning  the  phenomenon  of 
picture  superiority  in  memory,  existing  theories  agree  that  the  use  of  graphics 
facilitates  memory  and  learning. 

Motivation  &  Attention: 

Learning  from  graphical  information  is  an  interactive  process  of  perceiving, 
integrating,  and  reconstinicting  information  from  the  given  materials  (Fitzerald, 
1989).  How  knowledge  is  constructed  depends  upon  how  information  interacts 
with  learners*  expectations  and  intents,  and  their  motivation  to  learn.  Emotion 
affects  how  information  is  attended  to,  stored,  and  retrieved  (Langhinrichsen  & 
Tucker,  1990;  Lehmann  &  Koukkou,  1990).  Since  perceiving  information  involves 
selecting  and  filtering  the  elements  from  given  stimuli  (Pettersson,  1989),  the 
learners'  interests  and  expectations  play  an  important  role  in  extracting 
information  and  reconstructing  their  understanding.  Viewed  from  the 
perspective  of  dual  coding  theory,  motivation  refers  to  the  arousal  and  goal- 
oriented  aspects  of  behavior  that  accompany  the  verbal  or  nonverbal  memory 
system  (Paivio,  1983, 1986). 

Rogers  (1983)  considers  emotion  as  a  third  cognitive  code  in  addition  to  verbal 
codes  and  imagery  codes.  He  argues  that  how  people  feel  is  related  to  how  people 
can  learn  and  remember  information.  What  makes  information  learnable  is 
largely  a  matter  of  coherence  to  one  s  beliefs  and  desires.  Implied  from  Rogers' 
interpretation,  the  emotional  effects  that  graphics  can  provide  are  highly  related 
to  how  pictures  are  processed  and  how  they  interact  with  learners'  internal 
mental  states.  As  proposed  by  Rogers  (1983),  affective  reactions  to  graphic 
information  is  part  of  the  encoding  process  in  the  cognitive  system.    He  assumes 
that  when  affective  responses  are  activated,  retention  is  enhanced.  A  positive 
attitude  toward  learning  materials  motivates  learners  to  operate  on  and  to 
organize  the  given  representations. 

Research  findings  suggest  that  graphics  create  memorable  impressions  and 
have  an  emotional  impact  that  might  affect  people  s  attitudes  toward  the 
information  described  in  the  printed  materials  (Samuels,  1970;  Swell  &  Moore, 
1980).  In  computer-assisted  instruction  materials,  still  or  animated  graphics  are 
often  used  for  purposes  of  gaining  attention  and  appeal  (Rieber,  1989a).  The 
instructional  intent,  however,  is  not  just  to  provide  enjoyment,  but  also  to 
encourage  cognitive  curiosity  and  learning  involvement.  As  involvement  is 
enhanced,  it  becomes  easier  for  students  to  acquire  concepts  from  verbal 
materials  provided  in  the  instruction  (Rigney  &  Lutz,  1976). 

Given  that  affective  reactions  to  pictures  ordinarily  occur  more  quickly  than  to 
words  (Paivio,  1986),  more  precautions  and  considerations  should  be  taken  in 
relating  what  is  perceived  by  learners  and  what  is  desired  to  be  perceived  from  the 
intended  objective,  Fitzgerald  (1989)  argues  that  the  use  of  pictures  needs  to  be 
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guided  by  the  properties  in  the  pictures  relevant  to  establishing  intended  belief 
and  desires  from  learners.  Because  different  people  have  different  experiences 
with  graphics,  when  a  picture  is  used  to  communicate,  consideration  should  be 
taken  regarding  whether  the  picture  can  encourage  learners  to  relate  desired 
reactions.  For  example,  the  use  of  red  color  in  a  drawing  might  create  a  warm 
feeling.  However,  red  color  might  also  represent  a  warning  or  wrong  response. 
The  design  decisions  about  integrating  specific  graphic  attributes^  such  as  color, 
format,  or  animation,  depend  not  only  on  which  attribute  to  choose  to  present  to 
learners,  but  also  how  tx)  use  it  properly,  and  when  to  use  it  to  meet  intended 
purposes* 

Empirical  Findings  of  Graphic  Elffects: 

Empirical  comparisons  of  gi'aphics  with  non-graphic  instruction,  or  one 
format  of  graphics  with  another,  have  been  conducted.  To  test  the  effects  of 
graphics  in  computer  assisted  instruction,  Alesandrini  and  Rigney  (1981) 
compared  all-verbal  to  verbal-graphics  presentations  in  a  program  with  96 
undergraduate  participants.  The  results  favored  the  verbal-graphics 
presentation.  Another  experiment  in  the  same  study  was  conducted  to  test  the 
effectiveness  of  the  pictorial  review  task  compared  to  a  read-twice  control.  The 
results  from  post-test  showed  that  the  pictorial  review  condition  facilitated 
performance  more  than  re-reading,  and  produced  more  favorable  attitudes  as 
well. 

It  has  been  reported  that  information  presented  with  graphics  facilitates 
comprehension  and  enhances  learning  of  science  concepts  (e.g.,  Arnold  &  Dwyer, 
1975;  Rigney  &  Rutz,  1976).  Reid,  Briggs,  and  Beveridge  (1983)  suggested  that  the 
readability  of  textual  materials  should  be  related  to  the  content  of  pictures 
accompanying  the  texts.  To  examine  the  effect  of  pictures  upon  readability  of  a 
specially  written  science  topic,  a  study  was  conducted  among  students  in  reading 
biology  materials.  The  findings  supported  the  use  of  pictures  as  a  factor  in  the 
long-term  understanding  and  recall  of  the  text. 

Although  evidence  has  been  provided  by  some  research  that  graphics  increase 
learning,  the  support  for  using  graphics  in  instructional  materials  is 
inconsistent.  In  a  reading  comprehension  study,  Edyburn  (1982)  indicated  no 
significant  increase  in  performance  for  a  graphic-CAI  group  compared  with  a 
non-graphic-CAI  group.  In  learning  comma  usage,  the  results  of  Smith's  study 
(1985)  showed  that  a  graphic  treatment  was  not  significantly  better  than  a  non- 
graphic treatment.  In  math  leaming,  Caputo  (1981)  concluded  that  graphics 
enhanced  the  learning  of  math  skills.  In  speUing  leaming,  Surber  &  Leeder 
(1988)  reported  that  provision  of  graphic  feedback  in  CAI  materials  did  not 
enhance  either  achievement  or  motivation.  From  the  inconsistent  results  among 
those  studies,  it  appears  that  related  variables  such  as  variation  in  learners' 
abilities,  differences  in  the  nature  of  the  learning  tasks,  and  design  quality  might 
need  to  be  considered  in  the  assessing  the  effectiveness  of  graphics. 
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To  examine  the  instructional  value  of  different  types  of  graphics  in  facilitating 
learners'  achievement  of  different  educational  objectives,  Berry  and  Dwyer  (1982) 
studied  subjects  of  different  levels  of  ability.  Graphic  treatments  included:  one 
control  group  viewing  instruction  with  no  visual  illustration,  one  treatment  group 
viewing  instruction  with  black  and  white  shaded  line  drawing,  another 
treatment  group  with  realistic  colored  shaded  line  drawings,  and  another 
treatment  group  viewing  non-realistic  colored  drawings.  Analyses  indicated  that 
subjects  of  different  levels  of  ability  profited  differently  from  identical  types  of 
visualization  and  that  identical  types  of  visualization  were  not  equally  effective  in 
facilitating  subjects'  achievement  of  different  educational  objectives. 

To  determine  the  effect  of  pictures  on  learning  to  read,  Samuels  (1970)  found 
that  there  was  no  difference  in  learning  between  pictiire  and  no-picture 
conditions  for  better  readers.  However,  among  poor  readers,  the  presence  of 
pictures  influenced  learning  a  sight  vocabulary.  He  assumed  that  the  less  able 
readers  were  using  a  strategy  of  looking  at  a  picture  as  a  cue  when  having 
difficulty  identifying  a  word.  This  finding  suggests  that  students'  ability  is  an 
important  consideration  in  determining  the  value  of  graphics. 

Reid  and  Beveridge  (1986)  studied  the  relationship  between  picture  facilitation, 
children's  ability,  and  text  difficulty.  Learning  for  picture  and  non-picture 
groups  was  measured  by  a  criterion-referenced  objective  test.  The  results 
indicated  no  general  motivational  effect  of  pictures  on  the  learning  of  text.  With 
students  of  higher  ability  level,  pictures  are  beneficial,  while  with  less  able 
students,  they  distract.  Apparently  higher-ability  students  can  extract 
appropriate  information  from  pictures  more  easily  than  less  able  students. 
Although  this  finding  contradicts  Samuel's  conclusions  (1970),  the  importance  of 
the  students'  ability  in  the  content  area  and  their  general  reading  ability  were 
addressed  in  both  studies. 

In  a  multifactor  analysis  of  the  instructional  effectiveness  of  self-paced 
visualized  instruction  on  diflerent  educational  objectives,  Dv/yer  and  Parkhurst 
(1984)  studied  the  effectiveness  of  different  types  of  graphics  when  supplementing 
programmed  instruction.  They  were  interested  in  how  students'  reading 
comprehension  level  influences  their  ability  to  profit  from  visual  instruction.  The 
results  of  this  study  indicated  that  different  types  of  visualization  used  to 
supplement  programmed  instruction  were  not  equally  effective  in  helping 
students  master  different  objectives.  They  also  found  that  the  achievement  of  the 
high  reading  comprehension  group  improved  as  the  degree  of  realism  in  the 
graphics  increased;  whereas,  the  achievement  of  students  in  low  reading 
comprehension  gi'oups  decreased  as  realism  in  the  graphics  increased.  A 
similar  conclusion  was  reached  in  Parkhurst  and  Dwyer's  study  (1983)  when  they 
assessed  students'  IQ  level  and  their  ability  to  profit  from  visualized  instruction. 

The  effects  of  graphics  in  assisting  learning  might  vary  when  the  formats  of 
pictures  are  difTerent.   In  teaching  wave  motion.  Gage  (1989)  used  three  different 
formats  of  computer  graphics  among  three  treatment  groups.  The  three 


324 


treatment  groups  include  the  most  realistic  graphics,  intermediate  level  realistic 
graphics,  and  the  least  realistic  graphics.  The  results  showed  that  the  treatment 
with  the  least  realistic  visuals  materials  had  the  highest  gain.  Realism  of  the 
graphics  materials  did  not  add  to  learning  in  this  study.  An  important 
implication  for  designing  computer  graphics  is  that  the  main  function  of 
graphics  is  to  help  students  to  differentiate  the  relevant  cues  in  a  graphic  display. 
Realism  may  actually  interfere  with  the  transmission  of  information  because 
irrelevant  cues  might  also  be  activated  in  processing  the  information. 

In  a  study  designed  to  evaluate  the  instructional  effect  of  realistic  and  non- 
realistic  graphics  (in  both  the  presentation  and  retrieval  of  information)  and 
s<-.udents'  levels  of  dogmatism,  college  students  received  a  dogmatism  pretest  and 
K,^^  then  assigned  to  different  graphic  treatment  groups.  Results  indicated  that 
a  significant  interaction  was  found  between  students'  relative  level  of  dogmatism 
and  their  ability  to  learn  from  different  color  cued  visual  materials  (Berry  & 
Dwyer,  1983). 

From  the  studies  cited  here,  it  can  be  seen  that  individual  differences  in  ability 
and  learning  experiences  interact  with  different  types  of  visual  illustration  for 
specific  instructional  objectives.  To  design  effective  graphics  used  in  com.puter 
assisted  instruction,  it  is  necessary  to  investigate  the  ability  level  and 
characteristics  of  learners,  and  match  graphics  with  learning  tasks  and 
objectives.  For  example,  should  visual  displays  be  simple  or  more  complex,  what 
colors  and  cues  should  be  used,  and  in  what  sequence  should  these  graphics  be 
presented.  Additional  questions  related  to  students  cognitive  abilities  and  to 
learning  outcomes  should  be  addressed  in  the  design  and  use  of  graphics. 

Computer  graphics  have  played  an  important  role  in  the  way  learners  interact 
with  electronic  texts  (Reinking,  1992).  Especially  in  designing  science  or 
mathematics  instruction,  mandatory  viewing  of  static  or  animated  gi-aphics 
becomes  necessary  to  perform  intended  learning  tasks.  How  well  students  can 
learn  from  graphics  might  be  determined  by  how  well  students  can  interact  with 
the  graphic  materials.  In  problem  solving  programs,  Thomas  (1989)  used 
computer  graphics  together  with  problem-solving  activities  for  improving  student 
achievement,  their  attitudes  and  motivation  in  learning  functions  and 
transformational  geometry.  One  of  the  objectives  of  the  study  was  to  evaluate  the 
effects  of  graphics  problem  solving  activities  on  the  achievement  and  attitudes  of 
secondary  mathematics  students.  It  was  found  that  students  who  were  allowed  to 
select  a  picture  to  work  with  throughout  the  activities  resulted  in  higher  levels  of 
motivation  than  students  who  were  allowed  to  change  goals  throughout  the 
activities.  It  was  impUed  that  providing  different  access  conditions  for  allowing 
interaction  with  pictures  might  influence  the  level  of  understanding  of  the 
concepts  presented  through  the  graphics.  To  facilitate  deeper  processing  in 
learning  from  graphic  information,  the  instruction  should  also  provide 
opportunities  for  sufficient  interaction. 
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Four  experiments  involving  180  undergraduate  students  were  designed  to 
evaluate  the  effectiveness  of  computer  graphics  when  learning  algebra  word 
problems  (Reed,  1985).  The  purpose  of  the  study  was  to  explore  the  conditions 
under  which  computer  graptdcs  could  be  used  to  improve  students'  estimates  of 
average-speed  problems.  It  was  observed  that  effective  use  of  graphics  required 
learning  by  coaching,  especially  for  poorer  achieving  students. 

Learning  physics  usually  requires  learners  to  visualize  the  motion  and 
trajectory  of  objects  in  order  to  construct  concepts.  To  study  how  animated 
graphics  interact  with  learners  and  assist  learning,  Rieber  and  his  colleagues 
conducted  various  animation  studies  among  different  grade  levels  of  physics 
students  (Rieber,  1989b,  1990a,  1990b;  Rieber,  Boyce,  &  Assad,  1990;  Rieber  & 
Hannafin,  1988).  As  expressed,  animated  presentations  help  students  visualize 
those  physical  laws  which  involved  changes  in  speed  and  the  path  of  travel. 
However,  the  optimal  use  of  animated  graphics  is  related  to  practice  activities. 
Although  pictures  might  promote  deeper  levels  of  mental  processing  during 
encoding,  the  absence  or  ineffectiveness  of  practice  might  provide  little  facilitation 
of  memory.  The  effective  use  of  computer  graphics  depended  upon  supportive 
instructional  activities. 

Design  ImplicatJ.ons: 

This  review  of  the  literature  suggests  that  the  effective  use  of  graphics  depends 
upon  student's  cognitive  ability,  differences  in  learning  tasks  and  objectives,  and 
other  supportive  instructional  strategies.  These  estimations  can  guide  decisions 
about  when  to  use  graphic  information  and  how  to  use  it.  These  design  decisions 
may  influence  how  learners  interact  with  the  graphics.  They  also  influence  how 
leaiiiers  understand  and  retain  concepts  extracted  from  the  representations. 
Prior  to  designing  instruction,  analysis  of  the  intended  cognitive  tasks  and  the 
cognitive  ability  of  learners  becomes  an  important  task. 

Fi^om  reviews  of  the  various  graphics  research  studies,  several  principles  for 
guiding  graphics  use  can  be  tentatively  stated.  The  principles  are  predicated  on 
the  notion  that  to  design  effective  graphics,  carefully  considering  the  learners  and 
what  they  are  to  leam  is  needed  in  addition  to  considering  the  graphics 
themselves.  Some  guidelines  can  be  summarized  as  follows  (examples  for  each 
guideline  are  provided  in  Appendix  1): 

1.  Graphics  should  be  used  to  guide  learners  to  the  important  information 
rather  than  distract  learners  to  focus  on  irrelevant  information. 

2.  Graphics  should  be  relevant  to  the  learning  objectives  and  the  ideas  to  be 
communicated  in  the  text. 

3.  The  graphic  objects  used  should  be  concrete  and  meaningful,  with 
consideration  of  learners'  familiarity  in  specific  area, 

4.  The  use  of  graphics  should  consider  how  learners  interpret  the  graphics 
given,  and  their  ability  to  peruse  the  materials  for  performing  the  required 
interactions. 
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5.  The  use  of  certain  graphic  objects  should  encourage  learning  interests  of 
specific  learner  groups,  and  be  appropriate  to  their  characteristics. 

6.  The  level  of  graphic  realism  required  usually  depends  upon  what  types  of 
information  need  to  be  conveyed,  how  much  information  is  to  be  given,  and 
how  much  time  is  provided  for  conveying  the  information. 

7.  The  design  format  should  be  concrete  and  explicit  for  the  concepts.  When 
an  analogy  is  used,  experiences  with  the  analogy  used  must  be  considered. 

8.  If  too  much  information  is  included  in  a  picture,  it  is  necessary  to  break  it 
into  smaller  or  simpler  units  and  present  one  unit  at  a  time. 

9.  If  graphics  contain  concentrated  information,  it  is  necessary  to  provide 
visual  cues,  such  as  colors,  labels,  or  arrows,  to  emphasize  the  important 
information  that  shoiild  be  focused  upon. 

10.  The  use  of  graphics  should  encourage  positive  learning  attitudes. 

Siunmary: 

There  are  a  variety  of  sound  reasons  to  use  graphics  in  computer-based 
instructional  materials.  We  have  reviewed  some  of  these  reasons  from  the 
cognitive  and  affective  perspective  by  looking  at  existing  theory  and  research. 
Rapid  advance  in  computer  technology  has  made  possible  an  increasing  array  of 
sophisticated  and  appealing  computer  graphics.  Design  trends  should  reflect  not 
only  the  interest  of  graphic  artists  but  also  the  needs  of  learners.  Adding 
aesthetic  dimension  to  designing  CAI  can  substantially  encourage  people's  active 
involvement  in  learning  materials.  However,  designers  need  to  be  cautious  about 
the  individual  differences  in  observing,  interpreting,  recalling,  and  encoding 
information.  Different  kinds  of  pictures  facilitate  learning  in  different  ways.  The 
use  of  graphics  should  not  just  be  based  on  a  designer's  intuition.  Instead,  how 
viewers  interpret  the  pictorial  information,  and  what  can  be  done  to  bridge  the 
gap  between  what  learners  know  and  what  learners  need  to  know  should  be  a 
primary  concern. 

Guidelines  based  research  findings  are  limited  in  current  research  on 
graphics.  Most  studies  are  done  in  a  specific  field-   Instead  of  investigating  a 
narrow  range  of  application,  future  studies  on  graphics  area  should  consider  a 
wider  range  of  variables  and  application.  To  ensure  that  designers  use  graphics 
more  effectively,  guidelines  must  be  generated  from  an  understanding  of 
cognitive  and  affective  phenomena. 
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Appendix  1:  Examples  of  Design  Guidelines 


L  Graphics  should  be  used  to  guide 
learners  to  the  important 
information  rathCT  than  distract 
learners  to  focus  on  irrelevant 
information* 

The  graphic  in  this  example  is 
used  to  obtain  attention  from 
viewers  about  the  problems  that 
might  happen  to  their  own  cars. 
The  infomiation  follows  when 
the  graphic  is  displayed. 


2.  Graphics  should  be  relevant  to  the 
learning  objectives  and  the  ideas 
to  be  communicated  in  the  text. 

The  graphic  in  the  example 
represents  the  job  of  a 
veterinarian.  The  holding  hand  in 
the  graphic  infers  the  meaning  of 
"protecting"  which  captures  the 
role  of  a  veterinai  ian. 


Veterinarians  are  responsible  for  the  velfare 
of  our  nation's  food-prodi^'cing  and  companion 
animals,  a  ro^e  crucially  important  to 
maintaining  public  health. 
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3.  The  graphic  objects  used  should 

be  concrete  and  meaningful,  with 
consideration  of  learnCTs' 
familiarity  in  specific  area. 

4.  The  use  of  graphics  should 

consider  how  learners  interpret 
the  graphics  given,  and  their 
level  to  peruse  the  materials  for 
performing  the  required 
interactions. 

When  simple  line  drawing 
symbols  are  used,  labelling  them 
with  what  they  represent  might 
help  to  avoid  confusion  or 
misunderstanding. 
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.  The  use  of  certain  graphic  object 
should  encourage  interests  of 
specific  learner  groups,  and  be 
appropriate  to  their 
characteristics. 


Graphics  that  use  cartoon 
characters  might  interest  your^i 
learners.  However,  whether 
cartoons  provide  the  same 
interest  for  older  learners  should 
be  carefully  considered. 


,  The  level  of  graphic  realism 
required  usually  depends  upon 
what  types  of  information  need 
to  be  conveyed,  how  much 
information  is  to  be  given,  and 
how  much  time  is  provided  for 
conveying  the  information. 

The  picture  on  tlie  left  might  be 
sufficient  for  achieving  objective 
1:  Identify  the  eyeball. 
However,  it  might  not  be 
detailed  enough  to  help  achieve 
objective  2:  Identify  the  pupil, 
eye  lid,  and  eyelashes 


Objective  1 


Objective  2 


.  The  design  fonnat  should  be 
concrete  and  explicit  for 
presenting  key  concepts.  When 
an  analogy  is'used, 
considerations  about  learners' 
experiences  with  the  analogy  are 
required. 

The  graphic  example  tries  to 
communicate  that  time  is  not 
what  money  can  buy. 


u  12  1 
10      ^  2 
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$10000000000000000000 


Hov  much  IS  a  second  vor^h  ? 
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If  too  much  information  is 
included  in  a  picture,  it  is 
important  to  dissect  it  into 
smaller,  simpler  units  and 
present  one  unit  at  a  time. 

For  example,  in  teaching 
physiology  or  a  related  content 
area,  dissecting  the  information 
into  digestible  pieces  is 
necessary.  It  might  also  be 
helpful  to  show  the  part  and 
whole  relation  through  the  use  of 
a  cropping  technique. 


Cereberal  Cortex 


Brm  Stem  Centers; 
Pons   


1  1^1  mii^tmd 

Medulla  Oblongata 


9.  If  graphics  contain  concentrated 
information ,  it  is  necessary  to 
provide  different  visual  cues, 
such  as  labels,  colors,  or  arrow 
cues,  to  emphasize  the  important 
information  that  should  be 
^ocused  upon. 

In  the  example,  the  label  cues  and 
arrow  cues  are  used  to  identify 
the  portions  that  nee4  to  be 
described  in  details. 


Kidney 


bladder 


^Heart 
Liver 


[Reproductive  orgarT 


10.  The  use  of  graphics  should 
encourage  positive  learning 
attitudes. 

In  the  example,  the  graphic 
feedback  is  to  encourage  learning 
interest. 


A 

Congratulations! 

You  solved  5  problems 

correctly. 

©©©  I 

©©  i 
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COURSEWARE  EVALUATION:  TECHNIQUES  FROM  INSTRUCTIONAL  DESIGN  THEORY 

AND  COGNITIVE  SCIENCE 

Dale  Y.  Hoskisson 
Valley  City  StalcUniversity 

Introduction 

The  computer,  like  any  other  tool,  is  only  as  valuable  as  the  software  and  the  skill  and  knowledge  of 
the  user  allow  it  to  be.  It  is  important  then  that  we  have  good  programs  to  use  on  our  schools'  computers. 
However,  There  is  a  widely  felt  concern  about  the  quality  of  coursewaic  available  for  schools  (Allesi  and 
Trollip,  1991;  Hannafin  and  Peck,  1988;  Krendl  and  WilUams,  1990;  Price,  1991;  Vcnczky  and  Odin, 
1991).  In  our  efforts  to  improve  the  quality  of  courseware  it  would  be  useful  to  have  a  tool  to  describe 
instructional  systems  without  reference  to  media  or  thcoiy.  Such  a  tool  would  be  valuable  for  two 
reasons.  First  it  would  give  designers  a  way  to  see  clearly  what  they  have  created  and  help  catch  many 
problems  at  an  early  stage.  Second  it  would  give  the  consumer  a  powerful  tool  for  obtaining  information 
for  evaluating  courseware.  With  this  information  the  consumer  would  be  able  to  make  judgments  about 
what  program  to  buy  that  would  be  based  on  the  instructional  qualities  of  the  programs. 

This  paper  explains  some  components  of  a  taxonomy  that  is  being  developed  at  the  Center  for 
Systematic  Coursewaixi  Integration,  Design,  and  Evaluation  (C-SCIDE)  to  descnbe  instruction  for  use  in 
evaluating  educational  software  (Hoskisson,  1991,  1992a,  1992b).  The  elements  of  the  taxonomy  that 
have  been  developed  so  far  include: 

•  an  objective  classification  system 

•  terminology  for  describing  instruction 

•  a  method  of  mapping  instructional  interactivity 

This  paper  will  briefly  discuss  the  objective  classification  system  and  then  do  a  more  in-depth 
explanation  of  some  parts  of  the  terminology  to  describe  insti'uction. 

Objective  Classification 

In  order  to  design,  study,  or  evaluate  instructional  systems  it  is  helpful  to  know  what  the  objective  is. 
If  a  comparison  study  is  being  done,  it  is  necessary  to  know  what  the  objective  of  each  system  is  to  see  if 
they  have  the  same  objective  and  to  measure  their  success.  When  we  evaluate  instmction  we  must  know 
what  the  intended  and  actual  objective  are.  Dciei*mining  the  objective  is  one  of  the  first  steps  in  designing 
instruction.  Therefore  we  need  a  way  to  classify  objectives  according  to  teaming  outcomes.  Theiu  arc 
many  schemes  available  (Bloom,  1956;  Gagnd,  1985;  Kylloncn  and  Shutc,  1987).  The  choice  of 
taxonomy  is  somewhat  arbitrary.  I  have  chosen  Merrill's  (1983)  taxonomy  (  see  Figure  1)  because  I  find 
it  is  easy  to  use  without  losing  accuracy.  The  taxonomy  consists  of  two  dimensions,  performance  and 
content. 

The  performance  dimension  has  three  levels: 

•  Remember  is  a  level  that  asks  the  student  to  ^produce  or  recognize  some  item  of  information 
from  memory.  This  is  the  only  level  that  deals  with  facts.  Therefore  the  find  and  use  levels  for 
facts  are  shaded  in  Figure  1. 

•  Use  is  the  level  that  asks  the  student  to  apply  an  abstraction  to  a  particular  case. 

•  Find  is  a  level  that  requires  the  student  to  originate  or  invent  a  new  abstraction. 

The  second  dimension  consists  of  four  kinds  of  content: 

•  Facts  are  pieces  of  information  that  are  arbitrarily  connected  such  as  proper  names,  dates  or 
events,  the  names  or  symbols  of  places,  objects,  or  events. 
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Procedures 
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Type  Of  Content 
Figure  1.  Objectives  classification 


•  Concepts  are  groups  of  symbols,  objects,  or  events  that  share  common  characteristics  and  the 
same  name. 

•  Procedures  arc  organized  sequences  of  steps  to  accomplish  a  particular  goal. 

•  Principles  explain  or  predict  v^hy  things  happen.  They  represent  causal  or  correlational 
relationships  that  interpret  events  or  circumstance 

A  fuller  explanation  can  be  found  in  Merrill,  1983. 


Teminology  to  describe  instruction 


Merrill  (1983)  has  developed  a  set  of  primary  and  secondary  presentation  forms  or  instructional 
characteristics.  We  have  adapted  these  for  the  taxonomy.  The  primary  forms  consist  of  two  levels  of 
specificity  and  two  modes  of  p'Xisentation.  The  two  levels  of  specificity  are  general  and  instance. 
Material  can  be  presented  as  a  generality  such  as  an  explanation  of  the  commutative  property  of 
multiplication  using  x  and  y.  Material  can  also  bo  presented  as  specific  instances  such  as  using  the  word 
church  to  demonstrate  the  digraph  *'ch." 

The  two  modes  of  presentation  arc  expository  or  telling  and  inquisitoiy  or  asking  (Merrill,  1983). 
The  levels  of  specificity  and  the  modes  of  pi-eseniations  are  combined  into  expository  generality  (EG), 
expository  instance  (Ecg),  inquisitory  generality  (IG),  and  inquisiiory  instance  (leg). 

The  secondary  presentation  forms  includes  most  of  Memll's  original  set  (1983)  and  several  others  we 
have  added.  Those  secondary  presentation  fornis  that  we  have  retained  from  Merrill  are 
•    feedback  (correct  answer) 
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♦  representation  (present  intbrmation  in  alternate  representation) 

♦  mathemagenic  help  (underlining,  bold  type,  etc.) 

♦  mnemonic  (memoiy  aids) 

♦  prerequisite  (help  recall  prerequisite  learning) 

♦  context  (place  the  new  information  in  context) 

We  have  added  several  others  based  on  the  work  of  West,  Farmer  and  Wolf  (1991).  They  arc: 

♦  chunking 

♦  concept  mapping 

♦  advance  organizers 

♦  analogy/metaphor 

Their  are  two  two  types  of  chui\king  or  organizing  su'ategies  that  can  be  used  in  insu*uction.  The  first 
type  is  linear/spatial  strategies.  This  type  includes  spatial,  sequential,  procedural,  and  logical.  The 
second  type  is  classification  which  includes  taxonomies,  typologies,  and  multipurpose  sorting. 

Spatial  strategies  organize  knowledge  according  to  relationships  in  space  such  as  dividing  the  states 
into  western  states,  states  east  of  the  Mississippi,  or  the  Great  Lakes  states.  Sequential  strategies 
organize  knowledge  according  to  relationships  in  time  such  as  the  break  down  of  communism  in  Eastern 
Europe  or  the  events  of  the  Kennedy  assassination  and  the  shooting  of  Oswald.  Procedural  strategies 
organize  knowk  ige  according  to  relationships  of  stages  such  as  a  recipe  or  the  steps  to  dry-mount  a 
picture.  Logical  strategies  organize  knowledge  according  to  relationships  that  can  be  logically  induced 
or  deduced  such  as  proof  for  a  theorem. 

Taxonomies  are  used  to  organized  knowledge  that  can  be  described  by  logical,  law-like  relationships 
such  as  instructional  objectives  or  the  animal  kingdom.  Typologies  show  less  logical  order  but,  never  the 
less,  are  based  on  easily  obsei'vcd,  obvious  features  such  as  color,  size,  or  personality  type.  Multipurpose 
sorting  can  bo  done  according  to  cause-effect,  similar-different,  form-function,  or  advantage- 
disadvantage. 

West,  Farmer,  and  Wolf  (1991)  describe  concept  mapping  as  a  "way  of  graphically  displaying 
concepts  and  relationships  between  or  among  concepts  (p.  93)."  The  map  is  a  visual  portrayal  of  how  the 
concepts  are  related  to  each  other.  A  learning  hierarchy  could  be  displayed  as  a  concept  map  as  could  an 
instructional  design  system  model. 

Advanced  organizers  serve  as  a  bridge  or  connecting  link  from  known  infonnation  to  unknown 
information.  The  connection  is  the  similarities  between  the  known  and  the  unknown.  An  advanced 
organizer  is  a  brief  prose  passage  that  recalls  the  known  information  and  the  aspects  that  are  similar  to  the 
unknown  information.  The  similarities  are  explicitly  stated  and  the  new  information  is  outlined.  A 
statement  recalling  the  Yankee  reasons  for  the  Revolutionary  War  and  outlining  how  they  are  similar  to 
those  of  the  wars  for  independence  in  South  America  would  be  an  advanced  organizer  for  a  unit  on  the 
South  American  independence  movement. 

Metaphors,  analogies  and  similes  are  another  type  of  bridge  or  connecting  link  from  known  to 
unknown.  They  are  figures  of  speech  that  create  a  comparison  of  two  unlike  things  but  which  have  some 
characteristics  in  common.  Comparing  an  instructional  objective  to  a  target  is  a  metaphor. 

Application  of  the  Taxonomy  to  Two  Sample  Courseware  Products 

We  have  been  using  the  taxonomy  for  a  year  now  and  have  found  it  a  valuable  tool  in  our  evaluation 
of  courseware.  We  use  several  methods  of  recording  the  descriptions  of  software.  One  method  is  a 
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checklist  of  the  instnictional  characteristics  of  a  program.  Figure  2  shows  the  results  of  using  the 
checklist  to  describe  'The  Great  Solar  System  Rescue"  from  Tom  Snyder  (1992).  The  instructional 


Name  of  Program 

Type  of  Program 

Subject 

Great  Solar  Svstem  Res, 

game 

Astronomy/solar  system 

Tom  Snyder  Prod. 

Objective;  Use/concept 

Students  use  given  information  to  determine 

where  an  object  is  in  the  solar  system. 

Presentation  forms 


EG 

Beg 

y 

IG 

leg 

y 

Representation 

Matiiemagenic  help^ 

Mnemonic  ^ 

Prerequisite 

Context 

Chunking 

y 

Concept  mapping 

Advance  organizers 

Analogy 

Figure  2.  Checklist  for  Great  Solar  System  Rescm 


characteristics  of  the  program  arc  uncomplicated.  Information  is  presented  in  an  expository  (Eeg)  manner 
and  a  question  (leg)  is  asked.  The  information  is  chunked  to  help  in  applying  it  All  of  this  is  embedded 
in  a  game  format  the  characteristics  of  which  arc  not  shown  on  the  checklist. 

Figure  3  shows  a  checklist  for  ''The  Puzzler"  from  Sunburst  (1986).  The  checklist  shows  that  this  is  a 
simple  program.  The  program  presents  (Eeg)  a  story  fragment  and  asks  the  learner  to  make  predictions 
(leg)  based  on  the  fragment.  After  the  learner  makes  one  or  more  predictions  the  program  presents 
another  fragment  of  the  story  and  repeats  the  process.  There  is  no  correct  answer  provided  because  it  is 
intended  to  be  an  open  ended  exercise  and  allow  students  to  fully  develop  their  own  thought  processes. 

These  analyses  done  with  the  taxonomy  are  not  complete  by  themselves.  There  are  many  qualities  of 
the  programs  that  arc  not  captured.  How  aJl  these  characteristics  interact  with  each  other  is  not  shown. 
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Name  of  Prograir     Type  of  Prograir.  Subject 


The  Puzzler 

drill/practice 

reading  comprehensioi 

Sunburst 

Objective:  find/concep 
Students  need  to  use  given  information  to 
make  predictions  about  aspects  of  a  story. 

Presentation  forms 


EG  Chunking 

E^g  y       I  Concept  mapping  j  

 I      Advance  organizers  

leg  y       I  Analogy  

Representation 
Mathemagenic  help 
Mnemonic 

Prerequisite  

Context  I 

Figure  3.  Checklist  for  The  Puzzlei 

Neither  is  the  sequencing  shown.  For  these  and  other  needs  we  have  developed  additional  elements  of  the 
taxonomy  (Hoskisson.  1991, 1992a,  1992b). 
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Computer-based  Ibols  for  Methodology  teaching 


P.  Pintelas      A»  Kameas     Af.  Orampes 


Abstract 

In  tlus  session,  we  address  the  problem  of  the  training  of  company  employees  in  project  culture,  and 
specially  in  the  use  of  methodology.  At  first  we  present  MEDOC,  a  whole  life-cycle  methodology  tliat 
may  be  applied  in  the  management  of  projects  in  various  domains  of  the  software  industry.  The  structure 
of  tliis  HKthodology  is  suitable  for  instruction  through  a  computer-based  training  tool.  Such  a  tool  is 
IvIETHODMAN  I,  which  represents  the  commercial  version  of  an  Intelligent  T\itoring  System  for  basic 
methodology  training.  METRO DMAN  I  had  originally  been  developed  as  a  prototype  aimed  at  testing  die 
efficiency  of  a  certain  padagogical  approach  in  teaching  methodology.  To  meet  the  user  requirements  that 
emanated  from  METHODMAN  I  usage,  we  are  currently  developing  GEPRIAM,  a  highly-interactive  Au- 
thoring Envirormient  supporting  die  construction  of  iiitelUgent  methodology -teaching,  computer  b;iscd 
applications.  GEPRIAM,  in  order  to  improve  system  mwDdularity,  uses  a  niathodology  ezpert  system  caid 
a  pedagogical  expert  system,  which  assist  the  user  during  application  construction  and  execution.  Rnally, 
the  airrent  status  of  GEPRIAM  development  is  evaluated,  and  some  considerations  for  future  upgrade 
are  presented. 

1  Introduction 

Since  technological  progress  and  innovative  technologies,  and  the  associatc^d  benefits  for  all  IcvcU 
of  modern  enterprises  [Gurbaxani,  1991],  are  usually  available  to  or  accessible  by  all  enterprises, 
the  difference  lies  in  the  degree  of  adaptation  of  the  enterprise  to  the  new  technologies,  or  in  the* 
i-ate  of  adoption  of  the  new  technologies  by  it;  these  both  depend  critically  on  the  people  that  will 
introduce  or  use  the  new  technologies. 

Therefore,  both  personnel  and  managers'  development  and  training  arc  two  of  the  key  issut^s 
in  todays  bussincss  world  [Friis,  1990].  Personnel  is  a  very  important  parameter  in  competition. 
Capable  project  managers  may  develop  products  correctly  and  within  budget  [Sommerville,  1989], 
but  in  order  to  develop  advanced  products  that  use  innovative  technology  yet  are  beneficial  for 
the  investor,  well- trained  project  managers  are  needed. 

As  managers  and  industriaUsts  develop  an  appreciation  of  the  significance  of  training  people  in 
the  xise  of  methodologies,  the  production  and  dissemination  of  products  for  niethodolody  education 
and  training  is  rapidly  gaining  ground.  The  current  situation  is  that  training  disciplines  rarely 
include  education  in  project  culture.  Such  an  education  should  include  the  introduction  both  of  a 
project-type  organizational  structure  (responsibilities,  knowledge,  controls  etc)  and  of  the  use  of 
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methodology,  and  sessions  in  risk  management  (technological,  human,  etc).  This  lack  frequnetly 
caiist^s  the  feiilure  of  innovative  projects  and  also  hinders  the  dissemination  of  new  technologies. 

Up  to  now,  people  were  trained  in  courses  controlled  and  carried  out  by  other  people  (the 
experts).  This  practice  involves  a  high  cost  in  time,  money  and  spirit,  since  the  trainees  had  to 
be  moved  to  the  training  site  and  the  expert  had  to  be  constantly  in  his  best  shape,  in  order  for 
the  training  session  to  maintain  a  standard  level  of  quality.  Apart  from  depending  on  the  expert's 
mood,  success  of  the  training  also  depended  on  the  capability  of  the  trainees  to  absorb  information. 
Usually,  the  weaker  trainees  determined  the  pace  of  the  session.  Finally,  it  is  dear  that  using  this 
practice  it  is  almost  impossible  to  massively  train  a  large  number  of  people  efficiently. 

Consequently,  the  traditional  training  practices  can  hardly  meet  any  of  the  current  training 
requirements,  v/hicli  include  [Menu,  1990): 

©  rapid  development  of  student's  skills,  in  accordance  with  his  needs  and  time  schedule,  and  of 
course,  at  his/her  location 

•  adapting  the  sequence  and  pace  of  training  to  student's  personal  requirements 

•  preparing  and  assisting  the  student  before  and  during  the  session 

•  (enhancement  of  the  training  environment  and  of  the  communication  between  student  and 
tutor 

Tlius,  new  technologies  had  to  be  used,  and  especially  computers,  which  proved  very  success- 
ful in  this  field  due  to  fch(nr  capability  to  process  information  very  quickly,  an<l  to  "customize" 
tlioms<»lves  to  people's  particular  needs.  Computers  proved  very  efficient  especially  in  fields  where 
human  knowledge  v/cis  alrc^ady  cod(*d  in  a  structured  aad  "logical"  way;  such  was  the  case  with 
methodologies. 

In  the  next  section,  we  present  the  characteristics  a  methodology  suitable  for  computer-based 
training  should  have.  Such  a  methodology  is  MEDOC,  which  is  also  briefly  presented.  Then, 
we  analytically  present  METHODMAN  I,  a  computer-based  tool  that  teaches  the  adaptation  of 
MEDOC  in  software  project  management.  METHODMAN  I  had  originally  been  developed  as  a 
prototype  aimed  at  testing  the  efficiency  of  a  certain  padagogical  approach  in  teaching  methodol- 
ogy. To  meet  the  user  requirements  that  emanated  from  METHODMAN  I  usage,  GEPRIAM,  a 
highly  interactive  Authoring  Environment  supporting  the  construction  of  intelligent  methodology- 
teaching,  computer  based  applications  is  currently  being  developed;  GEPRIAM  is  briefly  present- 
ed in  the  next  section,  and  the  current  status  of  development  Ls  being  discussed. 

2    Methodologies  suitable  for  computer-based  teaching 

Although  a  human  tutor  a)uld  probably  teach  any  methodology,  this  sadly  is  not  yet  true  for  com- 
puterized tutors,  mainly  Ix^causc^  methodologies  axe  developed  by  humans  to  be  used  by  humans. 
Keeping  in  mind,  however,  that  it  is  humans  who  will  construct  the  computerized  tutors,  we  can 
state  a  set  of  characteristics  that  a  methodology  should  have  in  order  to  l>e  possible  to  effectively 
codify  it  into  a  computerized  tutor: 
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•  internal  sti^cture:  it  must  be  composed  of  well-defined  phases  and  subphases,  each  of  which 
is  an  integral  task,  with  well-defined  "execution"  semantics  (people  involved,  actions  to  be 
taken,  output  to  be  delivered  etc),  and  a  well-defined  set  of  preceding  and  succeeding  tasks. 
Phases  or  subphases  may  also  have  an  internal  structure.  This  decomposition  may  continue 
until  trivial  phases  or  tasks  are  reached. 

•  well-defined  goal  approaching  steps:  every  methodology  is  applied  to  solve  some  land  of 
problem.  The  procedures  that  solve  all  problems  must  be  well-defined,  in  the  iSense  that 
every  step  taken  must  be  justifiably  goal-approaching.  This  does  itot  mean  that  there  must 
not  exist  alternative  solution  paths  for  a  certain  problem;  it  means  that  the  structure  of 
methodology  should  be  able  to  "justify"  the  correctness  of  the  path. 

•  solution  and  procedure  documentation  and  justification:  apart  from  being  self-justifiable,  a 
methodology  must  be  capable  to  adopt  domain-depondent  explanation  for  the  procedures 
and  tasks  it  contains,  so  that  the  easy  justification  of  the  solution  reached  is  possible. 

•  generality  and  domain  independence:  it  should  be  possible  to  apply  a  methodology  to  similar 
problems  over  several  domains,  although  the  way  and  the  means  of  application  may  be 
different.  1£  this  is  not  possible,  then  an  attempt  to  extract  a  more  general  methodology 
should  be  made. 

•  flexibility  and  capability  to  evolve  over  time:  a  methodology  should  be  flexible  in  the  sense 
that,  if  needed,  it  could  be  customized  (to  a  certain  extend)  to  meet  personal  requirements 
without  losing  its  structure  or  effectiveness.  In  other  words,  different  people  could  have 
different  understandings  of  the  same  methodology,  without  this  fiict  affecting  the  methodology 
at  all.  In  addition,  a  methodology  should  evolve  over  time  to  capture  the  new  cases  that  may 
be  added  to  the  set  of  cases  it  can  deal  with. 

•  successful  application:  if  a  methodology  has  been  successfully  applied  to  several  domains, 
then  its  procedures  and  tasks  will  have  become  as  efficient  as  possible. 

The  first  three  characteristics  relate  to  the  procedures  a  methodology  adopts,  and  the  ca- 
pability to  justify  them.  This  will  permit  the  effective  monitoring  of  learner  progress,  and  the 
capability  to  present  and  justify  the  correct  answers.  The  fourth  characteristic  leads  to  formal 
olarity  [Clancey,  1987]  by  permitting  the  separation  of  domain  knowledge  from  teaching  strategies. 
Especially,  tiie  second  and  fourth  characteristics  relate  directly  to  the  two  levels  of  structure  that 
knowledge  has,  namely  surface  structure  and  cfeep  structure  [Yolles,  1991]  (which  result  to  the 
acquisition  of  surface  and  deep  knowledge,  respectively).  The  fifth  characteristic  tries  to  ensure 
that  a  computer  tutor  will  not  become  obsolete  after  a  short  period  of  time  (at  least  due  to  causes 
inherent  in  tlie  methodology).  The  final  characteristic  has  to  do  with  the  expertise  transfered  to 
the  computer  tutor  compared  with  the  expertise  needed  to  obtain  a  successful  result. 

2.1  MEDOC 

MEDOC  is  a  general  methodology  for  project  management  with  a  W(»ll-defined  internal  structure. 
It  divides  the  project  management  process  in  a  number  of  phases  [COMETT,  1991].  Each  phase 
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is  composed  of  task-conteni  and  task-control  activities.  The  former  define  tUe  steps  that  must  be 
carried  out  by  the  project  manager  in  order  to  complete  the  phase,  while  the  latter  define  the 
phase  terminating  activities  (usually  carried  out  by  the  inspectors).  The  project  manager  must 
talc(^  ivomc  action  (a)  to  coiiipletc  an  activity.  The  project,  as  W(?ll  as  the  activities  and  the  actions 
that  make  up  its  phases  produce  or  consume  information  products'^  these  have  the  meaning  of 
inputs  (prerequisites)  or  outputs  (objectives),  respectively,  of  the  action  or  activity  (for  example, 
documents,  reports,  meetings,  interviews,  etc).  The  sequence  of  phases  is  strict.  Each  phase  forms 
a  siibgoal,  which  may  be  achieved  by  following  the  activities  it  contains.  The  sequence  of  activities 
may  or  may  not  bo  strict.  This  is  not  the  most  flexible  structure  one  could  find,  but  it  is  a  working 
one. 

MEDOC  can  be  (and  has  been)  appUed  to  various  domains,  where  project  management  tasks 
exist.  Although  the  names  of  the  phases  or  activities  may  change,  the  structure  m  effect  remains 
the  same.  This  allows  for  easy  justification  of  actions,  or  sequence  of  activities  or  phases.  MEDOC 
is  in  use  for  several  years  now,  and  its  procedures  have  reached  a  point  of  liigh  effectiveness.  Its 
general  structure  permits  the  adaptation  of  the  methodology  to  all  the  problems  of  software  project 
management  that  may  come  up. 

hi  the  application  of  MEDOC  in  the  domain  of  software  project  management,  the  project 
management  process  is  considered  to  be  made  up  of  a  sequence  of  six  phases: 

1.  Starting  the  project 

2.  Rinctional  Analysis 

3.  Internal  Conception  /  Design 

4.  Implementation  of  the  modules 

5.  Integration 

6.  Testing  /  Acceptance 

Each  of  these  phases  is  made  up  by  a  sequence  of  task  content  activities  and  is  ended  by  a 
task- control  activity.  All  these  activities  entail  one  project  manager  action,  that  is  why  the  notion 
of  actions  is  not  explicitly  used  in  this  adaptation  of  MEDOC. 


METHODMAN  I  is  a  CAI  tool  which  aims  at  teaching  the  MEDOC  methodology  for  project 
management;  in  ])articular,  it  teaches  the  adaptation  of  MEDOC  in  the  domain  of  Software  Project 
Management. 

As  is  the  case  with  most  computer  tutors,  METHODMAN  I  is  composed  of  a  set  of  content 
files  and  a  set  of  support  programs.  The  latter  are  used  tx)  present  the  former  to  the  user,  and 
in  general,  these  make  up  the  invariable  part  of  the  application  (invariable  in  the  sense  that  if  a 
different  METHODMAN -like  application  was  to  be  constructed,  these  programs  could  be  reused, 
while  a  new  set  of  content  files  would  have  to  be  produced).  Execution  is  commanded  by  a  driver 


3    METHODMAN  I 


344 


5 


program,  which  reads  a  file  ^vith  the  teaching  strategy.  In  METHODMAN  I,  the  teaching  strat(>gy 
is  "prewired"  in  the  driver  program,  and  thus  the  sequence  of  execution  of  the  programs  used  to 
display  the  various  content  files  cannot  change.  This  was  a  design  decision  aiming  at  reUeving  the 
pedagogical  burden  jtom  the  authors  and  permitting  them  to  concentrate  only  on  the  application 
contents. 

3,1    Pedagogical  issues 

METHDOMAN  I  uses  the  Crampes  learning  cycle  [Crampes,  1991]  as  its  pedagogical  basis.  The 
Cramped  learning  cycle  (figure  1)  overcomes  the  problems  reported  in  [Cunningham,  1990]  and  is 
particularly  appropriate  for  complex  learning  material  [Yolles,  1991].  It  adopts  a  mixed  program 
control  method  (part  of  the  course  Is  learner-controlled,  and  part  of  it  Is  system  controlled), 
which  Ls  airrcntly  regarded  as  the  most  effective  teaching  strategy  [Bolton,  1991],  and  describes 
a  learn(T  in  a  learning  procc^ss  that  fest  passes  through  a  discovery  voyage*,  identifying  the  nature 
of  the  l(»arning  material  and  determining  its  form  and  structure.  Tlien  the  learner  examines  the 
content  and  explores  the  meaning  of  the  learning  material  through  an  exploration  stage.  Finally, 
evaluation  may  be  self-determined  or  objectively  done  by  the  system;  it  may  consist  of  a  sc^t  of 
tests  or  a  form  of  simulation.  E  evaluation  demonstrates  a  satisfactory  result,  then  the  cycle  may 
end.  Otherwise,  it  may  lead  to; 

(a)  a  poor  assessment  which  itself  can  encourage  or  demand  a  re-(* valuation  of  the  learning 
material  Jmd  continuation  of  the  cycle,  or 

(b)  (extended  discovery,  particularly  in  more  complex  learning  domains  which  have  more  than 
one  h\c\  of  perception  or  understanding. 

The  adaptation  of  the  Crampes  learning  cycle  to  the  needs  of  training  in  software  project 
manag(uucnt,  as  they  were  implemented  in  METHODMAN  I,  led  to  a  course  composed  of  four 
distinct  stages  [COMETT,  1990]: 

1.  Discovery,  where  the  trainee  is  presented  Avith  the  phases  and  the  activities  and  is  asked  to 
order  them  a^rrectly  (i.e.  to  discover  the  correct  evolution  of  a  project).  During  this  phase, 
the  methodology  expert  system  included  in  METHODMAN  I  is  monitoring  the  trainee's 
performance  and  guides  him/her  by  indicating  both  the  correct  answer  each  time  he/she 
makes  two  mistakes  in  a  row,  and  a  list  of  suggested  library  topics  that  he  should  consult. 
In  addition,  the  trainee  has  free  access  to  the  library  and  the  glossary  of  management  and 
informatics  terms. 

2.  Ebcploration,  where  the  tiainee  is  asked  once  again  to  correctly  line,  up  the  phases  and  cacli 
phjuse's  activities  of  the  project  life-cycle.  He  can  still  access  the  glossary  and  the  library, 
which  now  contains  tests  next  to  the  topics.  Wlien  the  trainee  successfully  passes  a  test,  he 
is  awarded  some  EKUs  (Elementary  Knowledge  Units),  the  number  of  which  depends  on  the 
typ(*  of  the  test  and  the  times  he  has  talcen  it.  Needless  to  say  that  every  mistake  is  paid  in 
vahiable  EKUs. 

3.  Evaluation,  where  the  trainee  passes  a  number  of  (admittedly)  difficult  tests,  with  his  EKUs 
score  decreasing  wth  every  mistake.  If  the  score  reaches  zero  before  the  stage  is  over,  the 
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trainee  is  thrown  back  to  the  previous  stage  and  is  asked  to  study  the  topics  more*  carc^fully.  If 
manages  to  finish  the  stage  with  a  positive*  EKU  score,  he  is  considered  to  have  acquired  the 
basic  methodology  in  software  project  management  and  is  awarded  a  password  that  aiables 
liim  to  enter  tlie  final  stage,  which  is  th(*  free  consultation  stage. 

4.  IVee  Consultation.  The  trainee  (now  an  acknowledged  project  manager)  can  now  review 
all  the  tests,  together  wth  the  correct  answers,  as  well  as  the  library  topics.  Note  here  that 
most  of  the  answ(*rs  to  the  tests  are  not  available  until  this  stage. 

3.2  Architecture 

The  functional  architecture  of  METHODMAN  I  is  depicted  in  figure  2.  The  general  idea  behind 
METHODMAN  I  is  to  incorporate  the  teaching  strategy  in  the  support  programs,  so  that  the 
au  \or  would  concentrate  on  the  structuring  of  the  content  to  be  taught.  This  content  is  structured 
in  (Cture,  text  and  test  frames.  There  exist  interpreters  that  display  each  of  these  types,  and  also 
ni£fc*iage  user -system  interaction.  The  pedagogy  of  the  application  is  incorporated  in  two  types 
of  stages  (each  of  which  is  used  twice,  resulting  to  a  four-stage  application);  the  order  of  stage 
execution  is  standard. 

The  overall  execution  of  the  program  is  controlled  by  the  driver^  which  reads  a  command  Une 
from  the  script  file  tliat  represents  the  pedagogical  content  of  the  application.  Depending  on  this 
line,  the  driver  activat(*s  the  appropriate  stage  monitor  program  with  a  set  of  appropriate  parame- 
t<*rs  indicating  the  stage  typ<\  The  driver  or  the  stage  monitor  may  also  activate  one  of  the  frame 
interpreters.  The  two  (xpcTt  systems  included  in  METHODMAN  I  also  play  a  very  important  role: 
the  methodology  expert  system  contains  a  set  of  rules  that  relate  to  the  structure  of  the  methodol- 
ogy phases,  the  library  topics  associated  with  each  phase  and  the  student  actions  (phase  selections 
or  test  answers)  that  the  system  regards  as  "correct",  as  far  as  the  methodology  is  concerned.  Tlie 
pedagogical  expert  system  creates  for  every  student  a  student  environment  composed  of  a  set  of 
files  that  contain  information  on  his  progress  in  the  course  and  his  performance  so  far,  using  the 
outcome  of  the  firing  of  the  nu^thodology  expert  system  rules  and  the  history  of  learner  system 
interaction.  The  structun*  of  the  expert  modide  of  METHODMAN  I  follows  a  cognitive  modelling 
^proach,  where  knowledg(*  Ls  decomposed  into  meaningful,  integratcvl  components,  and  thus  it 
may  be  easily  tmderstood  and  applied  by  humans. 

Although  METHODMAN  I  incorporates  an  expert  module  and  a  student  environment,  its 
interface  is  kept  as  simple  as  possible,  without  advanced  communication  capabilities  (like  direct 
manipulation,  multimedia  presentation  or  multimodal  communication).  In  addition,  its  teaching 
module  follows  a  script  where*  the  flow  of  execution  is  predefined  and  cannot  change.  The  task  of 
the  expert  systems  is  to  gtiid(*  the  learner  into  an  effective  sequence  of  methodology  topics  access, 
and  to  determine  whc^ther  the  k^arner  has  m(*t  the  pedagogical  objectives  of  the  oourse  at  the  end 
of  the  session.  Howev(*.r,  the  open  architecture  of  the  system  permits  tlie  integration  of  different 
support  programs,  and  in  such  away,  different  METHODMAN-like  appUcations  can  be  produced 
(using  a  set  of  independent  editors  to  produce  the  content  files)  that  will  match  the  continuously 
evolving  training  requirements. 
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3.3    Ease  of  use 


METHODMAN  I  offers  a  simple  yet  functional  User  Interface,  which  only  uses  some  function  keys 
(Fl  -  F5)  to  provide  access  to  its  utilities  (glossary,  library,  help  etc),  th(*  space  bar  and  the  Return 
key  to  continue  or  acknowledge  and  the  numeric  keypad,  the  T  key  or  the  F  key  for  answering 
the  tests.  Tlu^  \Lscr  can  scroll  within  the  menus  or  navigate  through  xhr  anbedded  melius  of  the* 
glossary  with  the  arrow  keys  (or  the  PgUp  and  PgDn  keys)  [METHODMAN,  1991]. 

In  addition,  METHODMAN  I  offers  five  significant  utilities  (avadlable  with  a  single  keypress) 
that  support  the  user  while  taking  the  course: 

1.  The  capability  to  exit  any  time  the  user  wants  and  to  restart  from  this  point  the  next  timc^ 

2.  A  Library  containing  nuni(>ruus  topics  on  software  project  nianageiuent  and  on  NIEDOC 

3.  A  Glossary  of  management  and  informatics-related  terms  asing  embedded  menus  and  win- 
dows to  move  around  related  terms 

4.  Up-to-date  progress  or  evaluation  test  score  (depending  on  the  pedagogical  context  of  user 
actions) 

5.  Online  help  on  the  system  functions  or  a  list  of  suggested  topics  for  the  trainee  to  read  (again, 
depending  on  the  pedagogical  context  of  user  actions) 

As  we  have  already  mentioned,  METHODMAN  I  is  a  very-easy -to  use  tool.  Therefore,  tliere 
is  no  need  for  the  user  to  be  trained  to  use  this  training  tool;  its  simple  interface  together  with 
the  on-line  help  make  it  easy  to- use  even  for  the  non  computer  expert. 

We  can  say  that  this  is  one  of  the  strongest  features  of  METHODMAN  I:  the  user  very  quickly 
gets  the  feeUng  he  is  in  control  of  the  process  with  no  additional  effort.  Wliile  he  is  running  through 
the  course  he  always  feels  "at  home"  with  it,  and  he  can  concentrate  his  efforts  in  absorbing  tlie 
knowledge  that  is  presented  before  his  eyes.  Let  \is  say  once  more  that  the  course  runs  in  the 
user's  own  pace;  in  addition,  the  user  is  allowed  to  interrupt  the  session  whenever  he  feels  tired, 
and  restart  from  that  point  the  next  time. 

3,4    METHODMAN  I  usage  modes 

METHODMAN  I  meets  most  of  the  requirements  that  a  user  may  liave  from  a  computer  based 
training  program  \vith  intelligence.  It  is  autonomous,  complete  with  respect  to  the  taught  method- 
ology, portable,  user  controlled,  and  easy-to  use.  More  specifically,  METHODMAN  I  can  be  \Lsed 
in  four  distinct  modes: 

1.  in  self-discipline,  where  a  user  runs  the  course  alone  to  improve  liis/her  knowledge  in  project 
management 

2.  as  help  to  instructors  before  or  during  a  training  session  in  the  use  of  project  management 
methodology 

3.  aii  an  assistant  to  project  leaders,  who  may  capitalize  upon  the  irvc  consultation  topics 
in  order  to  establish  the  habit  of  quality  and  methodical  work 
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4.  as  an  assistant  to  the  methodology  executive,  who  can  use  the  free  consultation  sheets 
and  the  multi-iiiothod  glossary  to  improve  the  Quality  Plan  of  the  company 


Tli(*  project  GESEM,  Ix^ing  nirrcntly  carried  out  hi  the  context  of  COMETT  II  program,  aims 
at  improving  and  ext(»n(ling  METHODMAN  I  by  delivering  a  package  of  advanced  training  tools 
[COMETT,  1991],  m  ovdci  to  introduce  the  project  culture  in  the  European  industrial  and  a- 
cadomic  communiti(*s  uivolved  with  innovative  activities  (those  aiming  at  developing  and  /  or 
iiitergrating  new  tt^cliiiolugies).  Directions  of  research  carried  out  by  the  GESEM  team  include 
those  proposed  in  [Clancey,  1990]  with  the  incorporation  of  a  more  powerful  pedagogic  expert 
system  and  the  adaptation  of  MEDOC  methodology  to  different  domains. 

Currently,  the  emphasis  is  placed  on  the  design  and  implementation  of  an  Authoring  Envi- 
ronment called  GEPRIAM,  which  will  aiable  the  authors  adapt  MEDOC  (or  any  well  structured 
methodology)  to  various  domains,  reducuig  the  effort  needed  to  cr(*ate  specialized  alucational 
packages  [Whiting,  1991], 

4.1    Tteaching  strategies 

An  application  construct(*d  ^vith  GEPRIAM  is  equivalent  to  the  sequential  execution  of  pedagog- 
ically  autonomous  obj(H-ts  we  call  learning  units.  Each  learning  unit  corresponds  to  an  integral 
block  of  knov/ledge  that  must  be  presented  to  the  learner  and  has  an  internal  structure  (simple 
or  complex).  Its  execution  is  pre-emptive  (in  the  sense  that  it  cannot  be  interrupted  by  the  exe- 
cution of  another  learning  imit)  and  has  dearly  defined  pedagogical  objectives  and  prerequisites 
(the  execution  of  a  learning  unit  cannot  commence  or  terminate  unless  these  objectives  are  met). 

These  objectives  may  corrc^spond  to  the  goals  being  sought  by  a  learner,  as  these  are  defined 
in  [YoUes,  1991].  Wli(»n  goal  reflated  objectives  are  not  clear,  the  learning  process  can  be  erratic 
and  mcfficient.  L<*arniiig  can  also  be  restrict(»d  by  other  factors  regarding  the  mental  mudol  of  the 
lcarn(^r.  That  is  why  difforcmt  k^arners  may  require  to  navigate  through  an  individual  sequence  of 
learning  units  in  order  to  achieve  a  particular  learning  goal,  since  learning  strategies  may  differ 
from  learner  to  learner.  To  support  different  learning  trajectories,  GEPRIAM  offers  a  variety  of 
teaching  strategies,  as  will  be  described  in  the  next  subsection. 

There  exist  two  types  of  learning  units: 

•  (^lementary  units,  such  as  text  or  picture  frames,  whose  internal  structure  is  almost  non- 
exixtent  (usually  they  are  referred  to  as  fi^ames),  and 

•  complex  units  called  stages,  which  encapsulate  a  teaching  method  and  consequently  have  an 
internal  structure  that  reflects  the  teaching  method  principles  (such  as  learning  by  discovery, 
by  tutoring,  by  examples  etc). 

In  GEPRIAM,  as  m  METHODMAN  I,  an  effort  has  been  made  to  relieve  the  author  from 
the  wcod  to  construct  and  si)(*cify  a  pedagogic  strategy  (our  experience  with  METHODMAN  I 
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showed  that  the  authoring  process  is  speeded  up  by  a  factor  of  20  in  such  a  case).  Stages  form 
a  set  of  predefined  pedagogical  strategies;  the  author,  who  need  not  be  a  pedagog,  may  dioose 
one  or  more  that  satisfy  liis  needs,  and  execute  them  in  some  order.  If  the  author  wants  to 
explicitly  construct  a  strategy  of  his  own,  tlie  system  supports  completely  this  phase.  It  has  been 
argued,  however,  that  to  achieve  effective  learning^  a  minimization  of  learning  constraints  must 
be  adopted  [Laurillard,  1987].  That  is  why  semi- structured  approaches  to  learning  can  be  more 
satisfactory  than  highly  structured  approaches.  In  GEPRIAM  stages  that  support  this  kind  of 
teaching  strategies  are  provided  to  the  author. 

GEPRIAM  supports  three  kinds  of  teaching  strategies  structuring: 

•  sequential  execution  of  learning  units:  here  the  teaching  strategy  is  encoded  in  tlie  stages 
used  by  the  author 

•  non  sequential  execution  of  learning  units:  hen*  we  have  two  levels  of  structuring:  the  jfirst  is 
within  the  stages  used  and  the  second  is  in  the  script  file.  The  learner  chooses  (or  is  forced 
to  follow)  a  path  from  a  set  of  allowable  paths  specified  by  the  author 

•  execution  of  the  learning  units  as  suggested  by  the  pedagogical  expert  system:  here  the  author 
defines  the  set  of  learning  units  and  the  pedagogical  prerequisites  and  objectives  associated 
>vith  each.  Then  the  pedagogical  expert  system  noanages  the  presentation  of  the  learning 
units  and  the  learner  interaction 

4.2  Architecture 

GEPRIAM  is  composed  of  two  major  subsystems:  the  authoHng  subsystem  and  the  user  execution 
subsystem  [COMETT,  1992].  The  former  is  used  for  application  construction,  while  the  latter 
supports  application  execution.  In  this  way,  a  prototyping  faciUty  is  provided  to  the  author,  while 
the  integration  and  packaging  of  tlie  application  is  facilitated. 

The  authoring  subsj  stem  is  composed  of  a  number  of  modules  that  enable  the  author  specify 
the  contents  of  the  application.  Although  GEPRIAM  is  not  yet  a  full  multimedia  environment, 
it  contains  a  text  and  a  picture  editor  (inclusion  of  a  sound  editor  ls  under  consideration).  A 
module  that  supports  the  construction  of  hypertext  glossaric^s  is  also  included.  Finally,  it  contains 
modules  that  support  the  specification  of  pedagogical  strategies  and  of  the  methodology  to  be 
taught.  The  output  of  the  authoring  subsystem  is  a  number  of  files  that  arc*  used  by  the  programs 
of  the  user  execution  subystem. 

The  teaching  strategy,  that  is,  the  learning  units  contained  in  the  application  and  all  the 
allowable  execution  paths,  is  contained  in  the  script  file  of  the  application.  The  execution  of  an 
application  is  commanded  by  the  application  monitor^  this  program  appropriately  calls  all  the 
other  programs  of  the  user  execution  subsystem.  These  are  mainly  learning  unit  interpreters^ 
that  is,  programs  that  display  the  different  types  of  learning  units  and  manage  learner  system 
interaction,  student  environment  programs^  which  monitor  the  progress  and  performance  of  the 
learner  within  the  appHcation,  and  some  utility  programs^  which  perform  supportive  functions,  like 
score  calculation,  application  exit  and  restoration  of  a  previous  position  etc. 
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GEPRIAM  aJs-^  includes  two  expert  systems: 

«  a  methodology  expeH  system^  which  is  used  during  the  authoring  process  by  the  author  for 
the  structuring  of  the  methodology  to  be  taught.  During  application  execution,  this  expert 
system  is  called  by  a^rtain  stage  types  to  evaluate  learner  perfomauce  and  guide  him  through 
the  correct  answer  path 

•  a  pedagogical  expert  system^  which  is  currently  used  only  to  evaluate  the  results  of  learner 
perfomance  using  the  data  a)llected  by  the  programs  of  the  student  environment.  When  com- 
pleted, it  wll  completely  support  learner -system  interaction,  and  thus  fecilitate  the  author 
in  the  specification  of  the  teaching  strategy 

Another  feature  of  METHODMAN  I  that  has  been  carried  over  to  GEPRIAM  is  the  open 
design  and  the  ability  to  replace  any  runtime  program  without  affecting  the  system  performance 
(for  ojcample  the  text  interpreter  may  be  replaced  with  a  new  advanced  version,  and  a  sound 
interpreter  can  be  included  as  well).  Compatibility  between  applications  created  with  different 
versions  of  GEPRIAM  can  be  maintained  by  maintaining  an  extended  set  of  runtime  support 
programs;  the  application  will  automatically  detect  and  use  the  appropriate  ones. 

4.3    Ease  of  use 

Rom  the  user  perspective,  GEPRIAM  has  Ix'en  designed  to  meet  several  requirements,  the  most 
important  of  which  being  to  impose  the  less  possible  cognitive  load  on  the  author.  To  this  end,  an 
advanced  user  interfaces  has  been  designed,  which  tried  to  make  an  efficient  use  of  screen  space,  so 
as  to  achieve  maximuia  communication  bandwidth  between  system  and  user  [Thimbleby,  1990], 
The  same  interaction  metaphor  has  been  used  in  all  the  modules  of  the  authoring  subsystem, 
with  the  left  part  of  the  screen  used  for  information  ejcchange  and  airrent  state  representation, 
while  the  right  part  of  tlie  sy,t(au  contains  a  set  of  "controls''  enabling  the  author  to  carry  on 
the  authoring  proce^ss  (th(»se  two  parts  may  also  be  interchanged  to  s*uit  the  needs  of  left  -handed 
users).  In  addition,  windows  are  used  to  separate  different  modes  of  operation,  and  menus  and 
buttons  to  present  \hc  author  with  the  available  actions. 

Two  more  features  that  lead  in  a  more  efficient  system-user  interaction  are  the  incorporation 
of  a  tightly  structured  model  of  iateraction  and  the  separation  of  the  cx)ntent  entry  phase  from 
the  teaching  strategy  specification  phase. 

Tlic  authoring  process  is  n^garded  as  the  satisfaction  of  a  series  of  imhgoals  which  amount  to 
the  overall  goal  of  application  construction.  A  pre-emptive  window  corresponds  to  each  subgoal. 
Within  each  window,  specific  arenas  are  used  for  smaller  siibgoals.  In  this  way,  the  user  knows  all 
goals  he  has  to  achieve  in  ordesr  to  construct  an  application.  Goals  are  achieved  using  actions]  the 
context  of  each  action  is  automatically  defined  by  the  subgoal  it  is  associated  with.  In  this  way, 
the  system  tacitly  guides  the  author  through  the  authoring  process,  and  it  is  also  flexible  enough 
to  let  him  control  the  process,  while  it  preveaits  him  from  pitfalls.  This  model  also  supports  the 
creation  of  learning  miit  libraries,  which  will  enable  the  easy  adaptation  of  MEDOC  to  various 
domains. 
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One  limitation  inherited  from  METHODMAN  I  is  that  the  author  cannot  yet  specify  the  iiser 
iiiterf  ice  of  tlie  application;  he  can  only  "use  the  one  supported  by  GEPRIAM. 

5  Conclusions 

In  this  paper,  have  pr(\scnted  two  CAI  tools  for  computerized  uu^tliodology  teaching  and  a 
suitable  methodology.  This  is  MEDOC,  a  general  methodology  for  project  management  \nth  a 
structure  that  permits  it  to  be  easily  coded  in  a  computer  program.  Such  a  program  is  METHOD- 
MAN  I,  the  first  tool  we  presented,  which  teaches  the  adaptation  of  MEDOC  in  software  project 
management.  The  architecture,  teaching  strategy  and  user  interface  of  METHODMAN  I  were 
presented,  and  its  weakiuosses  discussed 

As  part  of  the  product  mlidation  and  quality  assurance  plan,  some  demonstrations  were  i)er- 
formed  by  SYSECA  S.A.,  followed  by  evaluation  surveys  [SYSECA,  1989]  in  a  population  of  70 
users  (60  of  whom  were  not  familiar  mih  the  spirit  of  aiterprise).  The  outcome  of  these  surveys 
showed  that: 

•  82  %  were  generally  satisfied  with  the  program,  while  only  5  %  were  disappointed 

•  55  %  felt  that  their  knowledge  of  methodology  improved  a  little  after  a  session  with  METHOD- 
MAN  I,  while  45  %  said  that  their  knowledge  improved  a  lot 

a  81  %  would  recomnu^nd  METHODMAN  I  as  a  training  discipline  to  a  colleague  asking  for 
advice,  while  11  %  would  recommend  a  traditional  training  practice 

•  The  principal  quality  of  METHODMAN  I  is  xisefulness^  followed  closely  by  tiser-friendlincss 

Though  METHODMAN  I  is,  as  users  liave  sugg(»sted,  a  well-designed  tool  that  performs  sat- 
isfactorily [Kameas,  1992],  we  believe  tliat  it  could  be  improved  in  three*  directions,  in  order  to 
adapt  to  the  new  pcrsp(*ctives  and  techniques  that  are  shaping  the  field  of  computerized  education 
today  pea,  1987]: 

•  towaids  AI  by  evolving  into  a  full  InteUigent  T\itoring  System 

•  towards  multi-domain  adaptation,  by  providing  an  easy  ^vay  for  the  domain  expert  to  codify 
his  knowledge  in  METHODMAN-like  courses 

•  towards  nmltimedia  presentations 

To  this  end,  GEPRIAM,  an  Authoring  Environment  supporting  tlie  construction  of  method- 
ology teaching  applications  was  shortly  presented.  GEPRIAM  offers  many  supporting  functions 
and  an  advanced  user  mterface  to  support  the  authoring  process.  More  specifically,  the  author  is 
guided  through  the  phases  of  the  authoring  process  using  a  model  based  on  pedagogical  objectives 
and  learner  characteristics  to  specify  tlie  OTder  of  execution  of  the  course  learning  units.  Tliis 
model  regards  the  authoring  process  as  the  achievement  of  a  number  of  subgoals  that  lead  to  the 
course  construction. 
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Although  a  prototype  of  GEPRIAM  is  functioning  (as  part  of  system  validation,  METHOD- 
fvlAN  I  was  construct(^d  again  using  GEPRIAM),  the  system  is  not  yet  complete.  It  lacks  true 
multimedia  capabilities,  and  as  a  consequcnc(%  it  cannot  produce  multimedia  applications.  In  ef- 
fect, the  author  cannot  even  specify  the  user  interface  of  the  application.  In  addition,  a  large  part 
of  the  pedagogical  expert  system  has  not  yet  been  implemented,  and  learning  unit  libraries  are 
not  fiilly  supported.  Finally  the  student  aivironment  is  (^sc^ntially  the  same  as  in  METHODMAN 
I  and  needs  a  great  improvement. 

In  the  next  year,  wc  plan  to  use  GEPRIAM  to  aeate  several  applications  that  would  teach 
MEDOC  and  other  nu^thodologies  applied  in  various  domains.  By  evaluating  these  applications, 
wc  will  set  the  goals  that  the  next  version  of  GEPRIAM  will  have  to  meet.  Our  principal  aim 
will  be  the  construction  of  the  pedagogical  expert  system,  and  its  integration  with  the  system. 
Luckilly,  the  object-oriented  design  wc  have  used  will  greatly  facilitate  the  process. 

We  envisage  GEPRIAM  as  evolving  into  an  Intelligent  TUtoring  System  for  the  development  of 
intelligent  methodology -teaching  applications.  Our  effort,  however,  is  concentrated  in  providing 
as  many  facilities  as  possible  to  support  the  author  who  is  not  a  computer  expert.  V^'e  believe  that 
this  is  the  only  way  that  will  enable  the  efficient  coding  of  expert  knowledge  into  any  computerized 
teaching  tool. 
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Figure  2:  Functional  Ai-chitecture  of  Methodman  I 


Legem: 

Fl:  Pedagogical  Hxpen  System  (Manages  the  Student  Environment) 
F2:  Methodology  Expert  System  (Manages  the  Struaure  of  the 
Methodology  Presentation) 
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In  Search  of  New  IdeaSy  Research  Findings,  and  Emerging 
Technologies?  Here^s  Where  to  Find  Them 

Gary  C.  Powell^  M.A. 

The  University  of  Georgia 

(Paper  presented  at  the  34th  ADCIS  International  Conference, 
Norfolk,  VAy  USA,  November  8-11,  1992) 

Abstract 

There  are  many  avenues  available  to  CAI  practitioners  and  developers  in  search  of 
access  to  new  ideas,  research  findings  and  emerging  technologies  —  that  will  assist 
them  in  their  work  (developing  CAI),  Seven  such  avenues  are  discussed  in  detail. 
These  seven  avenues  are  as  follows: 

■  Graduate  Students  (Interns) 

"  Industry  Publications  &  Trade  Journals 

■  Industry/Trade  Conferences 

■  Direct  University  Education  (Graduate  Level  Courses) 

■  Hiring  Professional  Consultants 

■  Research 

■  Establishing  Affiliations  with  Universities 


Paper 

It  is  not  miusual  as  a  developer  of  computer  assisted  instruction  (CAI)  to  get 
extremely  bogged  down  with  work  assignments,  projects,  meetings  and  the  like.  I 
have  spent  many  hours  fiiUy  concentrated  on  something  as  minute  as  a  subroutine 
that  just  won't  work  right... and  before  I  knew  it,  not  only  had  lunch  come  and  gone, 
so  had  the  entire  day!  I've  also  found  myself  thinking  about  how  to  get  a  particular 
piece  of  code  to  work...  0/7  my  time  off  (good  grief)!  Anyone  who  has  had  a  tough 
time  seeing  past  "this  project"  has  consequently  felt  a  little  "out  of  touch"  with 
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what's  going  on  in  our  field  as  a  whole.  This  article  will  furnish  some  solutions  to 
our  problem. 

The  purpose  of  this  article  is  to  provide  and  discuss  seven  avenues  through 
which  CAI  developers  can  gain  ?*ccess  to  new  ideas,  research  findings  and  emerging 
technologies.  These  proposed  jlutions  will  assist  developers  in  their  work 
performance  (developing  computer-based  interactive  instructional  products).  These 
seven  avenues  are  as  follows: 

•  Graduate  Students  (Interns) 

■  Industry  Pubhcatiuns  &  Trade  Journals 
"  Industry/Trade  Conferences 

»  Direct  University  Education  (Graduate  Level  Courses) 

■  Hiring  Professional  Consultants 

■  Research 

"  Establishing  Affiliations  with  Universities 


1 .  Graduate  Students  (Interns) 

Graduate  students  (interns)  can  bring  unique  insights  to  corporate  organizations. 
They  are  usually  bright,  hard  working,  and  willing  to  leam.  Additionally,  graduate 
student's  minds  are  richly  soaked  with  theory  and  reseaich  that  CAI  developers  in 
the  field  may  be  missing,  as  the  developers  may  have  been  out  of  school  for  many 
years. 

Acquiring  interns  can  be  either  a  fonnal  or  informal  process.  Some 
organizations,  like  .Arthur  Andersen  &  Co.,  have  intemship  programs  that  are  very 
formal  and  structured,  and  hire  many  interns  (12-16)  three  or  four  times  a  year. 
Creating  such  an  arrangement  in  an  organization  requires  strong  "buy-in"  and 
support  from  both  upper-management  and  human  resources,  since  doing  so  is  a 
significant  undertaking  which  takes  much  commitment  and  effort  to  initiate. 
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Additionally,  a  constant  line  item  budget  allocation  for  the  internes  salaries*  will  be 
required.  Full-blown  internship  programs  take  time  to  get  established,  and  while 
they  can  greatly  provide  long  term  organizational  benefits,  it  is  a  major  imdertaking 
if  a  developer's  primary  concerns  are  his  or  her  personal  needs. 

With  some  other  organizations,  the  process  for  hiring  interns  is  not  as 
structured,  as  they  are  used  more  sporadically.  In  such  places,  hiring  an  intern  is 
more  "personal"  or  one-on-one  in  nature.  For  instance,  graduate  students  seeking 
an  internship  may  approach  professional  CAI  developers  or  managers  in  the  field 
personally  (e.g.,  they  send  a  letter  of  interest),  requiring  the  developer  or  manager  to 
provide  justification  for  hiring  the  intern.  Or,  if  a  need  has  already  been 
established,  the  CAI  professional  could  approach  an  Instructional  Technology 
department  at  a  local  university  (such  as  the  University  of  Georgia)  to  inform  the 
chairperson  that  they  are  looking  for  an  intern.  Since  only  one  or  two  interns  are 
being  dealt  with,  all  that  is  needed  is  the  approval  of  a  few  managers,  and  the 
approval  to  hire  a  "temp"  for  a  few  months.  This  "personal"  approach  is  a  quick 
way  to  get  in  touch  with  what's  going  on  in  the  field  of  technology-based  instruction, 
and  only  requires  the  developer's  own  commitment  and  some  extra  money. 

For  more  on  internships,  see  the  following  references: 

Couch,  R.  A.,  Branch,  R.  C,  &  Orey,  M.  (1989).  Corporate  internships  as  part  of 
a  graduate  program  in  Instructional  Technology.  Perfonnance  and  Instruction. 
28,(9),  26-28. 

Couch,  R.  A.,  Branch,  R.  C,  &  Orey,  M.  (1989).  A  systems  approach  to  selecting 
an  internship.  (ERIC  Document  Reproductive  Service  No.  ED  338  137) 

2.  Industry  Publications  &  Trade  Journals 

Industry  publications  and  trade  journals  (e.g..  Journal  of  Computer-Based 
Instruction)  provide  numerous  research  findings  and  reports  on  emerging 
technologies.  Taking  the  time  to  read  but  a  few  articles,  reports,  aiid  studies  on  a 

«       Note:  Not  all  internships  are  paid,  but  most  graduate  students  will  expect  to 
be  compensated.  The  salaries  for  interns  are  not  as  high  as  for  full  time  staff  (about 
1/2  to  2/3),  which  again  is  what  most  graduate  students  ai^e  accustomed  to. 
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regular  basis  can  provide  many  new  ideas.  Even  though  it  can  take  up  to  year  from 
the  date  an  article  was  written  and  submitted,  to  the  date  it  appears  in  print,  its  well 
woith  the  time  to  read  the  journals  and  newsletters. 

3.  Industry/Trade  Conferences 

Industry/trade  conferences  (e.g.,  ADCIS,  AECT,  SALT)  provide  a  plethora  of 
resources  from  which  to  get  new  ideas,  research  findings,  etc..  Beyond  attending 
the  concurrent  sessions  (which  is  paramount),  a  conference  is  a  place  to  establish 
dialogs  with  peers,  scholars,  and  other  professionals  in  the  field  of  C  AI.  By 
attending  sessions  and  networking,  one  can  get  a  good  grasp  on  whaf  s  going  on 
now.  It  cannot  be  stressed  enough  how  rewarding  it  is  to  attend  at  least  one 
conference  a  year.  There  is  much  to  be  learned  at  conferences,  both  formally 
(sessions)  and  informally  (socializing).  And  not  only  can  you  learn  fi-om  others, 
they  can  learn  from  you  too! 

4,  Direct  University  Education  (Graduate  Level  Courses) 

Direct  university  education  (graduate  level  courses)  via  a  local  university  will 
without  a  doubt  get  one  back  into  the  research.  Courses  that  may  be  relevant  to 
CAI  professionals  include:  computer-based  training,  computer  uses  in  education, 
artificial  intelligence,  video  production,  computer  graphics  and  animation, 
instructional  design,  human  factors,  adult  education,  cognitive  psychology,  and 
programming.  This  avenue  allows  developers  to  broaden  their  knowledge  of  the 
field,  and  make  them  more  valuable  to  their  employer  to  boot.  Attending  school 
part  time  allows  one  to  work  during  the  day  while  taking  classes  at  night.  Also,  it  is 
not  necessary  to  be  seeking  a  degree  (e.g.,  M.  A.  or  Ph.D.)  in  order  to  take  a  few 
courses.  Another  option  is  to  take  a  leave  of  absence  and  retiim  to  school  fiill  time. 
Those  who  are  lucky  work  for  companies  with  a  tuition  reimbursement  policy.  If 
not,  and  cost  is  an  issue,  consider  taking  one  or  two  classes  at  a  time;  or,  if  quitting 
work  is  a  prerequisite  to  attending  school  (jfull-time),  look  into  applying  for  a 
graduate  assistantship  that  off^ers  a  tuition  waiver  and  stipend. 
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5.  Hiring  Professional  Consultants 

Professional  consultants  also  can  be  a  rich  source  of  new  ideas,  new  ways  of 
doing  things,  and  emerging  technologies.  Since  consultants  move  around  from 
organization  to  organization,  they  tend  to  have  a  broad  perspective  on  specific  CAI 
techniques  (such  as  GUI's),  and  on  methods  and  technologies  that  other 
professionals  and  organizations  are  using.  Consultants  often  tend  to  know  first  hand 
what  works,  and  what  does  not;  thereby  giving  the  hirer  the  edge  of  avoiding  the 
mistakes  of  others.  Again,  as  v/ith  the  interns,  hiring  a  consultant  requires  "sign-ofF' 
fi'om  manager(s).  Consultants  can  be  costly  (much  more  costly  than  interns)  and  are 
generally  desired  for  specific  needs  (e.g.,  a  project  with  big  demands).  Unless  a 
project  requires  specific  skills  that  can  not  be  found  internally,  it  will  be  hard 
pressed  to  provide  justification  for  hiring  a  consultant. 

6.  Research 

Research,  that  is  actively  going  into  the  library  stacks  (as  opposed  to 
passively  subscribing  to  publications),  is  a  sure  fire  way  to  get  to  what's  going  on  in 
the  field.  Doing  research  is  not  an  arduous  task,  especially  if 'on-line'  or  CD-ROM 
platform  databases  like  ERIC  are  utilized.  Reseaixh  is  most  fiiiitfiil  when  there  is  a 
specific  topic  information  is  needed  on  -  like  instructional  models,  writing 
objectives,  or  developing  interactive  instruction  for  inner-city  youths. 

7.  Establishing  Affiiiations  with  Universities 

Establishing  affiliations  with  universities  requires  the  most  amount  of  work. 
Such  links  require  initiative  and  commitment  from  "up  top."  These  "high-tech 
connections"  between  industrial  firms  and  academic  institutions  require  a  substantial 
amount  of  cooperative  effort.  The  nature  of  the  affiliation  can  vary;  such  as  inviting 
college  professors  to  sit  on  an  academic  review  board,  or  providing  funding  to  an 
Instructional  Technology  (or  similar)  department/institute  in  exchange  for  theories, 
models,  design  documents,  instiTictional  techniques  and  actual  development  work. 
Affiliations  are  usually  long  term,  and  go  beyond  simply  "farming  out"  some 
development  work  to  some  graduate  students.  For  more  infonnation  on 
academic/industrial  relationships,  see  the  following  references: 
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Johnson,  L.  (1984),  The  High-Technology  Connection:  Academic/Industrial 
Cooperation  for  Economic  Growth,  ASHDE-ERIC  Higher  Education  Research 
Report  No.  6, 

Milligan,  F.  (1988).  The  vital  role  of  faculty  in  developing  successfiil  relationships 
with  business  and  industry.  (ERIC  Document  Reproductive  Service  No.  293  608). 

Powers,  D.,  Powers,  M.,  Betz,  F.,  and  Aslanian,  C.  (1988).  Higher  education  in 
partnership  with  industry.  San  Francisco,  London:  Jossey-Bass  Publishers. 

Schneiderman,  H.  A.  (1987).  Making  partnerships  work.  Across  the  Board.  24(5). 
28-29. 

As  can  be  seen,  there  are  many  options  available  to  CAI  practitioners  in 
search  of  avenues  through  which  they  can  gain  access  to  new  ideas,  research 
findings  and  emerging  technologies,  that  will  assist  them  in  their  work.  While  some 
suggestions  require  more  effort  than  others,  one  can  not  go  wrong  with  any  of  them; 
diligence  will  be  rewarded.  Both  you,  and  your  organization,  will  be  much  the 
better  off  -  so  go  for  it! 
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Interactive  Multimedia  Courseware: 
A  Perspective  on  Competency-Based  Training  for  the  Hospitality  Industry* 


Rod  Sims 
University  of  Technology,  Sydney 


Abstract 


This  paper  describes  an  Australian  project  which  has  developed  a  set  of  interactive, 
multinnedia  competency-based  courseware  modules  for  the  hospitality  industry.  The 
establishment  and  focus  of  the  project  depended  on  three  critical  factors:  receiving 
funding,  adopting  alternative  design  strategies,  creating  tpjly  interactive  courseware 
modules  and  integrating  testing  components  to  assess  competencies  attained.  The 
discussion  considers  initial  multimedia  impressions,  coping  with  demands  on  storage 
and  assisting  clients  new  to  interactive  technology.  Within  this  context,  the  acquisition 
of  appropriate  development  tools  and  hardware,  recording,  sequencing  and 
co-ordinating  audio  as  well  as  linking  multimedia  events  with  authoring  systems 
became  significant  considerations.  Other  areas  of  significance  included  project 
management  and  evaluating  learning  outcomes.  Experiences  gained  from  this  project 
were  invaluable  and  many  insights  into  acceptable  courseware  were  obtained. 


Introduction 

The  project  was  initially  conceived  to  develop  a  computer-based  training  system  for  the  Hospitality 
Industry.  However,  based  on  the  rapid  development  of  award-restructuring  and  the  demand  for 
multi-skilling,  the  project  was  expanded  to  include  a  prototype  management  system  to  support  the 
introduction  of  competency-based  standards.  Funding  was  applied  for  and  granted  by  the  Education  and 
Training  Foundation  (ETF)  of  NSW  in  July  of  1991,  and  the  specific  objectives  were  to  produce  a  range 
of  interactive  multi-media  courseware  modules  for  the  Hospitah'ty  industry  as  well  as  a  Competency 
management  system.  To  comply  with  ETF  funding  requirements,  a  number  of  organisations  were 
required  to  be  involved  with  the  project.  These  included  the  project  management,  design,  development 
and  marketing  groups,  associate  organisations  such  as  Tourism  Training  Australia  and  the  NSW 
Tourism  Industry  Training  Committee  and  supporting  hotels  and  clubs  which  undertook  to  provide  trial 
sites  anr'  pilot  study  subjects. 

The  Hospitality  Courseware 

The  courseware  modules  were  developed  using  current  trends  in  the  design  of  interactive  materials 
(Allen,  1992),  with  emphasis  on  successive  approximations  and  prototypes  rather  than  traditional  and 
linear  analysis-design-development  methodologies.  However,  considerable  effort  was  placed  on  delivery 

'    All  earlier  version  of  this  paper  was  presented  at  the  ITTE  Conference,  Brisbane  Australia,  September  1992. 
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strategy  and  interactive  sequences.  The  courseware  modules  selected  were  based  on  accredited  courses 
supplied  by  Tourism  Training  Australia,  and  designed  to  provide  the  foundation  for  the  common 
Introductory  and  Basic  Food  and  Beverage  modules.  Meetings  with  subject  matter  experts  and  industry 
representatives  ensured  that  the  specific  content  requirements  of  the  Hospitality  industry  were  met.  To 
evaluate  the  project,  selected  trial  sites  were  established  in  which  the  courseware  modules  were  installed 
to  assess  effectiveness  and  ease  of  use. 

A  major  consideration  was  the  delivery  platform  and  associated  development  tools  to  be  used.  After 
considerable  research,  it  was  decided  to  use  the  MS-DOS  environment  running  Windows  3.x,  as  most 
hospitality  organisations  had,  or  would  have,  similar  operating  platforms.  To  integrate  multi-media 
components  within  the  courseware,  the  decision  on  development  tools  was  based  on  experience  within 
the  development  team.  The  initial  decision  to  select  Authorware  Professional  for  Windows  was  based 
on  its  heritage  (a  descendant  of  PLATO)  as  well  as  its  productivity  and  flexibility  for  prototypes  and 
creating  complex  interactive  applications.  While  a  high  level  of  productivity  took  some  time  to  realise 
and  the  MS-DOSAVindows  multi-media  environment  was,  and  remains,  somewhat  restrictive,  the 
choice  has  been  justified  through  the  product  developed 


Courseware  Strategies 

The  major  strategy  used  for  the  courseware  modules  was  based  on  learner-integration,  proposed  by 
Sims  (1991,  1992),  which  suggests  the  presentation  of  interactive  events  in  which  the  student  is 
integrated  into  a  typical  work-environment.  By  providing  a  suitable  and  relevant  context  for  presenting 
content  material,  learning  outcomes  will  be  more  effective  than  those  presentations  which  use  the  screen 
as  a  surrogate  page.  The  observable  aspects  of  learner-integrated  courseware  are  a  series  of  displays 
which  provide  an  opportunity  for  students  to  interact  with,  and  possibly  suffer  the  consequences  of, 
manipulating  objects  within  their  work  environment.  The  learner-integrated  approach  is  an  enhancement 
of  many  simulation  strategies,  such  as  those  detailed  by  Alessi  &  Trollip  (1991),  and  represents  a  step 
towards  the  use  of  "virtual"  environments  in  interactive  training. 

One  of  the  major  requirements  for  implementing  this  approach  is  a  clear  definition  of  the  sequence  of 
events  which  is  to  take  place,  and  the  environment  for  their  action,  and  a  sample  of  the  descriptions  used 
for  one  of  the  Occupational  Health  and  Safety  modules  is  shown  in  the  Appendix.  As  an  example, 
Figure  1  illustrates  a  display  from  a  kitchen,  in  which  the  trainee  has  caused  an  accident  through  a  lack 
of  awareness  of  occupational  hazards.  This  particular  strategy  is  a  significant  departure  from  traditional 
courseware  applications  in  that  the  student  was  given  minimal  information  as  to  the  existence  of  hazards 
and  it  was  their  responsibility  to  identify  them  or  suffer  the  consequences  -  just  as  it  happens  in  the 
real  world!  Initial  feedback  from  the  trial  sites  suggests  that  this  strategy  has  been  successful. 

Similarly,  Figure  2  provides  an  example  of  one  of  a  series  of  actions  which  the  student  has  to  identify 
before  proceeding.  Again,  the  use  of  sequencing  based  on  hypothetical  real-life  events  provides  a 
sif^'ficant  departure  from  the  traditional  technique  of  presenting  individual  units  of  content  and 
illustrates  that  there  are  alternative  ways  in  which  the  technology  can  be  used,  especially  with  the  advent 
ol  high-resolution  graphics  and  sophisticated  development  tools.  This  use  of  alternative  presentation 
techniques  has  resulted  not  only  from  the  frequent  criticism  of  linear  courseware  but  also  from  the 
relative  newness  of  the  technology  itself  -  we  are  only  now  learning  about  its  most  effective  application. 
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Figure  I:  Learner-Integrated  Display 


Learner-Computer  Interface 


Recently,  significant  changes  in  courseware  applications  have  been  seen  in  design,  interface  and  control 
options.  The  traditional  format,  based  on  hierarchical  menu  systems,  demanded  the  availability  of 
extensive  controls  to  enable  the  student  to  move  "up"  and  "down"  the  hierarchy.  However,  this  approach 
made  it  very  easy  to  get  lost  in  the  menu  maze.  It  would  also  seem  that  this  structure  has  resulted,  in 
part,  from  the  application  and  interpretation  of  Instructional  Design  Methodologies  which  result  in  the 
content  hierarchy  Oeing  used  as  the  basis  for  lesson  design.  As  this  format  can  almost  eliminate  the 
capabilities  of  currenv  technology,  new  approaches  to  the  structure  and  design  of  courseware  were 
required.  The  following  aiscussion  examines  some  of  the  factors  which  have  contributed  to  this  change 
and  the  outcomes  seen  in  courseware  itself. 

Traditional  courseware  has  used  a  philosophy  based  on  the  trainee  being  at  one  "frame",  with  two 
general  options:  to  "go  forward"  or  to  "go  back".  However,  it  is  contended  that  there  is  a  fundamental 
flaw  in  this  approach,  based  on  both  human-computer  interface  and  life/work  patterns.  When  we,  as 
humans,  attend  training  or  perform  our  work  duties  (or  do  anything),  we  go  from  one  activity  to  another 
and,  because  time  travel  is  (so  far  as  we  know)  impossible,  we  are  not  able  to  return  to  our  earlier 
experiences.  However,  we  are  able  to  bring  with  us  into  the  present  objects  which  we  acquired  in  the 
past.  In  stark  contrast  to  the  general  way  we  operate,  courseware  development  projects  have  built 
complex  branches  and  paths  which  demand  trainees  to  manipulate  through  a  complex  hierarchy  in  order 
to  return  in  "time"  to  a  previous  frame. 

To  eliminate  this  problem,  a  new  environment  is  required  in  which  the  trainee  never  needs  to  return  to  a 
previous  screen,  but  operates  in  a  way  in  which  any  required  "objects"  are  transported  through  the 
lesson,  and  the  lesson  context  remains  as  a  real  environment.  Instead  of  returning  to  "Frame  X",  the 
trainee  can  return  to  a  building,  room,  desk  or  cabinet  and  retrieve  whatever  objects  are  desired.  Based 
on  this  argument,  it  is  suggested  that  courseware  should  mimic  our  real-life  patterns  and  behaviours 
rather  than  maintain  a  non-contextual  framework  v/ith  unrealistic  expectations  on  the  user  to  travel 
backwards  and  forwards  in  time.  This  concept  is  largely  based  on  the  model  provided  by  the  graphical 
adventure  games,  in  which  players  may  return  to  a  scene,  but  always  one  which  has  been  affected  by 
time  -  or  previous  visits.  From  another  perspective,  it  appears  that  players  of  such  games  have  little 
difficulty  learning  the  sequence  of  events  to  achieve  the  goals  -  is  it  not  possible  that  this  same  strategy 
could  be  used  for  learning? 

The  courseware  illustrated  in  Figures  1  and  2  provides  examples  of  this  concept,  and  additional 
illustration  is  manifested  in  the  concept  of  a  single-screen  delivery  environment,  with  all  controls  always 
available  to  the  trainee  (Figure  3),  providing  maximum  control  over  the  material  being  presented.  In 
many  ways,  it  is  analogous  to  a  TV  and  the  ability  of  the  viewer  to  select  channels  as  and  when 
required.  This  concept  also  places  the  student  in  complete  control  of  the  lesson  sequence,  although  there 
is  always  the  option  for  hidden  restrictions  to  be  built-in  by  the  developers. 

Assessment 

A  significant  component  of  the  courseware  development  was  the  provision  of  assessm.ent  modules  to 
cover  the  objectives  for  the  content,  as  specified  by  the  certifying  body.  For  the  modules  in  the 
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Designer;  additional  graphics  were  created  through  Core!  Draw  and  Windows  Paintbrush.  The  final 
items  were  all  converted  to  either  .PCX  or  .BMP  format  for  importing  into  Authorware.  The  graphics 
were  constrained  to  16  colours,  as  it  was  anticipated  that  the  installed  equipment  base  would  be 
operating  with  16-colour  VGA  monitors.  Given  the  current  installations,  this  was  a  conect  choice  and 
the  quality  of  the  courseware  has  not  been  reduced  by  these  colour  restrictions. 

The  basic  outcome  from  the  project  is  that  developers  of  multi-media  courseware  will  need  considerable 
expertise  in  creative  drawing  and  sketching  as  well  as  competence  in  the  complexities  of  the  various 
graphics  and  painting  applications  available.  However,  it  must  be  emphasised  that  creative  graphics  are 
only  one  component  of  the  courseware  product.  The  overall  quality  of  the  courseware  will  depend  on 
the  instructional  strategies,  contextual  structures  and  learner-computer  interface  concepts  implemented, 
and  which  are  represented  by  and  through  the  graphics. 

Creating  Audio 

The  audio  components  for  the  courseware  had  to  be  configured  to  the  Authorware  Professional 
requirements.  To  achieve  as  professional  quality  as  possible,  a  sound  engineer  and  sophisticated 
recording  equipment  (an  8-track  TASCAM  388  Studio  8)  were  used.  Using  both  live  (Foley)  and  CD 
sound  effects,  the  audio  segments  were  mastered  to  a  single  track  and  then  imported  using  the  VEDIT2 
utility  provided  with  the  Sound  Blaster  Pro  audio  card.  The  audio  was  recorded  to  memory  at  22MHz 
using  the  Lo  fitter  and  saved  as  a  .VOC  file  and  then  converted  to  .WAV  format  using  the 
V0C2WAV.EXE  utility. 

One  of  the  most  interesting  aspects  of  the  use  of  audio  is  the  increase  in  file-size  resulting  from  the  use 
of  sound  in  the  Authorware  lessons.  The  graph  in  Figure  4  illustrates  the  increase  after  the  sound  files 
were  included.  For  users  with  storage  capacities  of  80- 100MB,  this  can  begin  to  cause  problems  as  disk 
space  is  continually  have  to  be  found  to  store  the  applications.  As  a  guide,  it  is  suggested  that  new 
systems  have  a  200MB  hard  disk. 

Evaluation 

The  courseware  covering  The  Hospitality  Sector  has  so  far  been  installed  at  two  trial  sites  and 
evaluation  data  has  been  collected.  Overall,  the  feedback  has  been  very  pleasing,  with  almost  all 
respondents  indicating  a  positive  response  to  the  courseware,  as  indicated  in  Figure  5.  While  this 
positive  response  may  in  pail  be  due  to  the  novelty  of  the  material,  it  was  evident  that  the  provision  of 
self-paced  materials,  available  24-hours  a  day,  was  invaluable  for  organisations  whose  operations  and 
working  environment  is  such  that  staff  often  have  limited  opportunities  to  undertake  training  during 
their  shifts. 

A  more  important  aspect  of  the  evaluation  is  the  acceptance  of  multi-media  delivery,  which  is  reported 
as  providing  extremely  effective  learning  environments  (Larsen,  1992).  While  the  data  from  the  trial 
sites  has  provided  no  anti-media  evidence,  there  was  little  reference  to  the  audio  which  was  used  to 
support  user  understanding  of  the  material.  While  TV  and  radio  broadcasters  can  easily  specify  audio 
requirements,  courseware  developers  are  still  in  the  learning  mode. 
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Personal  attributes  required  for 
work  in  a  bistro  include  punctuality, 
flexible  hours  and  a  polite  nature 

The  bistro  environment  is  busy 
which  means  you  should  have  good 
health  and  a  positive  attitude. 


Z<Kirri 


1  /BIstKo  .| 

1  Canteen 

Casino 

Club 

Convention! 

Defence  j 

Education  j 

Fast  Food 

Hospital 

Hotel 

Motel  J 

Prison  J 

Private  j 

[  Relief 

Resort 

Restaurant 

Tavern  1 

Travel  1 

Award     |~^Ca'reer    |   ^^»^"''^"]||   j^^*^^^^    1     Size  | 


Figure  3:  Maximising  Learner  Control 


Hospitality  Industry,  the  assessment  questions  were  linked  specifically  to  the  units  and  elements  of  the 
competency  so  that  the  results  could  be  directly  accessed  by  a  management  system.  A  different  approach 
was  taken  with  the  second  module,  where  the  assessment  was  based  on  students  being  able  to  correctly 
work  through  the  Health  and  Safety  situations  without  prompts.  However,  the  challenge  was  to  provide 
assessment  items  based  solely  on  performance  criteria  (i.e.  competencies)  rather  than  knowledge. 

However,  the  assessment  of  competencies  through  interactive  media  is  a  complex  process,  especially 
when  the  courseware  is  focused  on  performance  factors  rather  than  content  structures.  In  terms  of  data 
collection,  this  has  been  one  of  the  most  complicated  factors  and  integration  with  the  management 
system  has  been  difficult,  although  once  implemented,  management  systems  provide  a  very  useful  set  of 
performance  data.  It  is  anticipated  that  the  courseware  and  assessment  modules  will  be  accredited  and 
employees  who  gain  achievement  status  in  selected  competencies  will  be  deemed  to  have  those 
competencies. 


Creating  Graphics 

Graphics  have  become  a  major  consideration  for  courseware  developers,  who  not  only  require  access  to 
sophisticated  graphical  software  but  also  the  services  of  competent  graphics  artists.  Based  on  the 
experiences  from  the  courseware  project,  graphics  were  defined  then  created  using  a  number  of  tools 
and  devices.  First,  sketches  were  hand-drawn  and  scanned  to  create  .PCX  and  .BMP  images;  these  were 
then  manipulated  using  Animator  Pro,  Windows  Paintbrush,  Windows  PaintShop  and  Micrographx 
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Figure  4:  Effects  of  Audio  on  Courseware  Size 
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Conclusions 


The  provision  of  interactive  training  facilities  for  the  introduction  of  competencies  into  the  Hospitality 
industry  has  proved  a  valuable  learning  experience  in  terms  of  the  development  of  multi-media 
courseware  and  the  integration  of  assessment  data  with  a  computer-based  management  system.  There 
are  many  skills  which  developers  of  traditional  courseware  will  have  to  learn  if  they  are  to  create 
effective  interactive  environments. 

Based  on  the  experiences  from  this  project,  it  would  seem  that  multi-media  has  not  only  provided  a  new 
set  of  tools  for  delivery  of  interactive  training,  but  also  the  opportunity  to  advance  the  effectiveness  of 
courseware  through  the  introduction  of  alternative  strategies.  These  strategies  make  better  use  of  the 
medium  and  focus  on  the  environments  and  performance  skills  with  which  trainees  are  familiar. 
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Appendix:  Course  Documentation 


Overview 

This  Appendix  provides  extracts  from  the  origituil  dociuvientation  used  to  describe  the  concept  aiid  logic  for  the  second 
Ho.-Train  interactive  module  on  Occupational  Health  and  Safety.  The  specifications  were  designed  as  a  storyboard  for  both 
the  graphics  artist  and  author  to  implement  with  Authorwaie*  Professionaf^  for  Windows.  It  should  be  noted  that  as  the 
product  was  developed,  a  number  of  changes  were  made  to  ensure  the  continuity  and  structure  of  the  interface  were 
maintained. 


Concept 

The  concept  of  this  lesson  is  based  on  a  "virtual  reahty"  instructional  game.  Tlie  aim  of  the  game  is  to  successfully 
maintain  a  safe  enviromnent  for  employees  and  apply  Orst  aid  techniques  where  necessary.  The  trainee  will  play  the  role  of 
a  new  Occupational  Health  and  Safely  Officer  in  a  Hotel.  The  conditions  of  the  game  are  such  that  the  trainee  must  visit 
each  of  the  available  scenarios;  until  they  have  completed  their  "jobs",  they  will  not  be  given  the  opportunity  to  review. 
However,  they  will  always  have  the  option  to  exit;  on  retuniirig  to  the  lesson,  their  data  should  be  maintained  and  the 
lesson  set  to  resume  at  the  point  of  exit. 

In  this  role,  the  trainee  will  be  asked  to  resolve  cenain  Occupational  Heahh  and  Safety  (OH&S)  issues  while  visiting  or 
working  in  each  of  the  scenarios.  For  each  scenario,  a  major  Accident  or  Event  will  occur;  in  addition,  the  trainee  will  be 
required  to  identify  approximately  three  potentially  dangerous  and/or  potentially  safe  working  conditions  in  each  scenario. 
If  these  conditions  are  identified,  the  trainee  will  be  rewarded  (Merit  Certificate)  and  ai\y  unsafe  conditions  corrected.  If 
the  conditions  are  not  identified,  the  trainee  will  be  shov/p.  the  results  of  not  correcting  an  unsafe  environment.  For 
example,  if  the  handles  of  pots  are  left  sticking  out  from  the  stove,  the  trainee  will  bum  themselves  and  incur  a  5-day 
"sick"  penahy. 

When  a  major  accident  occurs,  the  trainee  will  have  to  choose  the  correct  sequence  of  actions,  including  the  most 
appropriate  time  to  assist  injured  persons,  remove  dangerous  conditions  or  call  for  assistance. 

Scenario  Structure 

For  each  scenario  selected,  the  trainee  will  be  assigned  a  number  of  typical  hotel  jobs  to  complete.  They  will  have  the 
choice  of  one  or  more  of  (a)  completing  the  task,  (b)  identifying  dangerous  conditions  or  (c)  identifying  safe  conditions. 

If  (a),  the  u-ainee  will  receive  a  call  in  between  tasks  to  attend  another  incident.  If  (b),  the  trainee  will  be  asked  to  follow 
the  DR  ABC  procedure  as  required  as  well  as  contacting  the  appropriate  medical  or  managerial  staff.  The  selection  of  the 
correct  procedure  will  be  via  multiple  choice  icon-buttons  (e.g.  raise  arm,  bandage  woujid);  ea;  *i  button  should  be  logically 
valid.  At  the  end  of  each  major  accident,  the  DR  ABC  procedure  will  be  reviewed.  If  necessary,  the  trainee  can  ask  for 
assistance  from  a  supervisor;  if  this  occurs,  points  will  be  deducted.  ^ 

The  trainee  will  be  awarded  points  for  each  scenario,  depending  on  the  successftjl  identification  of  good/bad  conditions  and 
the  application  of  first-aid  support  to  injured  persons.  This  will  be  kept  as  a  running  total  on  the  "drop-down"  menu  bar. 
Trainees  will  be  introduced  to  the  option  to  "click"  on  potentially  dangerous  situations  without  being  prompted  -  at  start  of 
scenario  tasks,  trainee  can  select  a  new  task  or  click  to  correct.  Graphically,  the  "problems"  should  be  relatively  obvious. 

Overview  of  Kitchen  Scenario 

In  the  kitchen,  the  trainee  will  have  to  address  the  following  situations:  (a)  Chef  cut  by  slicer  OR  Attendant  cut  by  broken 
glass,  (b)  ICitchen  Hand  burnt  by  boiling  oil  OR  Frying  pan  on  fire,  (c)  Knives  not  put  away,  (d)  Badly  stored  items  and  (e) 
*Wet  floors. 
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Kitchen  Objects 

Pans  with  handles  sticking  out;  Pans  knocked  over;  Pans  with  handles  in;  Knives  lying  around;  Poorly  stacked  plates;  Pools 
of  water  on  ground;  Mop  in  comer;  Glasses  aroiuid  sink;  Electric  Sheer;  Deep  Fryer;  Broken  plates  on  floor;  Towels  and 
cloths 

Kitchen  People 

Chef;  Kitchen  Hand;  Maintenance  Man  on  ladder;  Chef  with  sliced  hand;  Kitchen  Hand  with  arm  badly  cut  by  glass; 
Kitchen  Hand  'Arith  oil  bum;  Close-up  of  lst/2nd/3rd  degree  bums;  Maintenance  Man  slipped  off  ladder;  Kitchen  Hand 
slipped  on  floor 

Kitchen  Tasks 

Washing  dishes;  Washing  pots;  Lifting  vegetables;  Cleaning  grill;  Stacking  Plates  and  Glasses 
Scenario  Logic 

One  point  scored  for  each  item  selected;  on  selection,  the  problem  is  rectified  and  the  danger  visibly  removed;  items  not 
selected  are  not  to  be  explained  but  recorded  for  later  use.  T  then  selects  task.  If  (a),  T  reminded  that  they  should  always 
check  for  any  potentially  dangerous  situations. 

Washing  Dishes 

T  shown  at  sink.  If  l(b)(iii)  recognised  then  message  to  indicate  they  have  done  well  and  possibly  avoided  an  accident.  If 
l(b)(iii)  not  recognised,  then  T  has  hand  cut  by  broken  glass,  not  badly  but  needs  attention  (this  results  in  a  loss  of  2 
work-days).  T  asked  to  apply  First  Aid  and  Safety  procedures:  (a)  remove  any  danger  (broken  glass),  (b)  don't  remove 
embedded  objects,  (c)  stop  bleeding  by  applying  pressure  (use  towels)  and  elevating  limb  and  (d)  contact  supervisor,  first 
aid  officer  or  ambulancCy  depending  on  severity. 
Use  a  place-in-correct-onler  sequence  by  point  and  click. 

Washing  Pots 

T  shown  at  sink  when  Chef  cuts  hand  severely  with  slicer,  T  must  follow  the  necessary  steps  to  provide  aid:  (a)  Assess 
scene  (stop  sheer),  (b)  Summon  supervisor/first  aider/ambulance,  (c)  Stop  bleeding  (pressure  with  towels;  elevate  limb),  (d) 
Sit  or  lay  patient  down,  (e)  Talk  and  reassure  patient  and  (0  Monitor  consciousness  -  if  drowsy,  lay  on  side. 

Lifting  Vegetables 

Before  commence  task,  frying  pan  catches  on  fire.  T  must  select  correct  fire  extinguisher.  Kitchen  Hand  has  bumt  arm. 
Actions  required:  (a)  Remove  patient  from  danger,  (bj  Summoa  supervisor/ambulance,  (cj  Cool  bum  with  cold  water,  (d) 
Re'  ^sure  patient. 

I/no  assistance  requested  for  lifting  (5  days  incapacitated),  damaged  back 
Cleaning  Grill 

OK  if  pan  handles  turned,  otherwise  T  bumt.  While  cleaning  grill  maintenance  man  carrying  ladder  slips  and  falls,  T 
notices  wrist  swelling.  Follow  with  procedure:  (a)  Sujnmon  assistance,  (b)  Rest  patient,  (c)  Apply  ice  compress,  (d) 
Support  bandage,  (e)  Elevate  and  (0  Consider  any  other  injuries. 

Stacking  Plates  and  Glasses 

OK  if  water  removed  from  floor,  otherwise  slip  and  fall. 
Summary 

Show  point  score  for  each  of  the  tasks  and  hazard  identification. 
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AN  INTELLIGENT  FRAMEWORK  FOR  THE  CREATION 
OF  EFFECTIVE  COMPUTER-BASED  INSTRUCTION 


J»  Michael  Specter 
Daniel  J.  Muraida 
Armstrong  Laboratory,   Brooks  AFB,  TX 

The  Advanced  Instructional  Design  Advisor  (AIDA)   is  an 
exploratory  research  and  development  project  at  the 
Armstrong  Laboratory  which  provides  on-line 
instructional  design  expertise  to  developers  of 
computer-based  training  (CBT)  materials  (Spector, 
1990) •     The  theoretical  framework  for  AIDA  consists  of 
second  generation  instructional  design  theory  (Merrill, 
Li,  6t  Jones,   1990),  the  nine  events  of  instruction 
(Gagne,   1985) ,  and  enterprise  theory  (Gagne  &  Merrill, 
1990) •     The  basic  idea  behind  AIDA  is  to  provide 
subject  matter  experts  with  an  intelligent  framework 
for  instructional  design  and  a  powerful  set  of 
development  tools.     This  paper  provides  an  analysis  of 
two  different  kinds  of  instructional  design  expertise 
which  have  been  prototyped  for  AIDA.     One  type  is  rule- 
based  and  the  other  is  case-based.     Our  findings 
indicate  that  the  case-based  methodology  shows  more 
immediate  promise,  which  supports  the  conclusion  that 
instructional  design  is  more  art  than  science. 

Several  researchers  have  attempted  to  develop  automated 
instructional  design  systems  which  guide  subject  matter  experts 
through  the  lengthy  and  complex  courseware  authoring  process 
(Duchastel,   1990) .     Enabling  subject  matter  experts  to  perform 
most  of  the  authoring  activities  associated  with  courseware 
development  has  two  distinct  advantages:     (1)  development  costs 
are  held  to  a  minimum,  and  (2)   instructors  are  empowered  to  make 
optimal  use  of  new  technologies.     The  primary  challenge  is 
capturing  and  making  available  (on-line)  meaningful  and  adaptive 
instructional  design  expertise. 

The  Armstrong  Laboratory's  Technical  Training  Research 
Division  has  an  exploratory  research  and  devr^lopment  project 
called  the  Advanced  Instructional  Design  Advisor  (AIDA)  which 
provides  subject  matter  experts  with  an  intelligent  framework  and 
a  powerful  set  of  object-oriented  development  tools  (Spector, 
1990)  .     The  theoretical  framework  consists  primarily  of  second 
generation  instructional  design  theory  (Merrill,  Li,  &  Jones, 
1990) ,  the  nine  events  of  instruction  (Gagne,   198  5) ,  and 
enterprise  theory  (Gagne  &  Merrill,   1990).     Enterprise  theory 
stresses  the  importance  of  an  integrated,  purposeful  human 
activity  as  the  focus  of  instruction  —  particularly  important  in 
the  task  analysis  phase  of  instructional  planning.     The  nine 
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events  of  instruction  provide  an  easily  accessible,  high-level 
framework  for  providing  instructional  design  guidance.  Second 
generation  instructional  design  theory  provides  the  concept  of  a 
transaction  shell  or  intelligent  framework  for  lesson 
development. 

The  objective-level  instructional  design  expertise  has  been 
prototyped  in  the  Experimental  Advanced  Instructional  Design 
Advisor  (XAIDA) .     XAIDA  makes  use  of  object-oriented  transaction 
shells  (Merrill  et  al.,   1990),  which  are  adaptable  and 
intelligent  lesson  development  templates.     The  larger  perspective 
(lesson,  module,  or  course)  instructional  design  expertise  has 
been  prototyped  in  the  Guided  Approach  to  Instructional  Design 
Advising  (GAIDA) ,  which  is  an  instantiation  of  the  nine  events  of 
instruction  with  four  completely  v/ork  case  examples.  Each 
approach  (XAIDA  and  GAIDA)  captures  a  different  kind  of 
instructional  design  expertise. 

Field  tests  indicate  that  both  kinds  of  expertise  are 
required  in  order  to  design  effective  CBT  (Canfield  &  Spector, 
1991;  Gagne,  1992;  Spector,  1990) e     Moreover,  there  is  growing 
evidence  that  a  great  deal  of  instructional  design  expertise  is 
not  easily  captured  in  the  form  of  procedures,  rules,  or 
algorithms  (Perez  &  Neiderman,   in  press?  Rowland,   1992;  Tessmer, 
1992) .     One  implication  of  this  finding  is  that  more  reliance 
should  be  placed  on  a  case-based  approach  to  instructional 
advising  when  attempting  to  provided  automated  instructional 
design  expertise. 


Intelligent  Design  Frameworks 

Thinking  About  Design  Guidance 

In  order  to  provide  a  context  for  offering  instructional 
design  expertise  to  novice  developers  of  CBT,  let's  first 
consider  an  analogous  situation  with  regard  to  architects.     It  is 
a  well-known  but  undocumented  fact  that  all  architects  and  most 
experienced  home  builders  are  fond  of  gold  nibblets.  Suppose 
that  gold  nibblets  have  been  discovered  in  Nebraska,  This 
announcement  has  caused  a  great  many  architects  and  builders  to 
move  to  Nebraska  thereby  creating  a  sudden  shortage  of 
architectural  and  home  construction  expertise  in  many  other  parts 
of  the  country.     A  forward  thinking  firm  in  Dime  Box,  Texas,  has 
decided  to  respond  to  this  crisis  by  building  an  architectural 
advising  program.     This  program  will  be  made  avrxlable  to  recent 
graduates  of  a  nearby  university  who  are  being  hired  by  local 
home  builders.     These  graduates  are  all  rednec\s  by  birth  and 
have  an  innate  understanding  of  hammers,  nails,   and  floor  joists. 
However,  these  novice  builders/architects  obviously  lack  the  kind 
of  expertise  in  design  and  construction  that  comes  from  years  of 
experience. 
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In  the  design  phase  of  this  effort,  this  firm  has  hired  an 
artificial  intelligence  expert  and  an  experienced  architect. 
They  have  provided  the  following  advice  regarding  the  home  design 
advisor  program: 

1.  There  are  four  kinds  of  advice  or  assistance  that  can  be 
provided  in  an  automated  home  design  advisor: 

a.  High  level  reminders.     These  are  things  that  should 
have  been  learned  :'n  school.     An  example  might  be  this:  If 
the  house  is  to  be  located  near  the  base  of  a  hill,  then  an 
underground  French  ditch  should  be  constructed  to  divert 
run-off.     It  will  be  important  to  make  this  kind  of  advice 
easily  accessible  and  context-sensitive  so  as  to  avoid 
information  overload.     On-line  help  systems  provide  a 
reasonable  model  in  this  regard. 

b.  Example  plans.     Tnese  should  be  based  on  previous 
cases  that  are  known  to  provide  attractive  (both 
aesthetically  and  economically)   solutions.     This  case-based 
advice  should  also  be  context-sensitive  and  easily 
accessible.     In  addition,  these  examples  should  be 
modularized  so  that  portions  could  be  extracted  and  inserted 
into  a  current  plan  under  development. 

c.  Configurable  shells.     These  provide  a  ready-made 
framework  for  the  kind  of  home  design  being  developed. 
These  shells  would  come  complete  with  default  settings  which 
would  determine  such  things  as  the  placement  of  doors, 
windows,  and  electrical  outlets.     Of  course,  the  defaults 
would  automatically  satisfy  existing  building  codes  since 
they  would  be  generated  by  a  rule-base.     These  shells  would 
provide  the  designer  with  a  rapid  prototyping  capability  so 
that  clients  could  be  shown  the  consequences  of  early  design 
choices  such  as  one-story  vs,  two-story,  adobe  vs.  wood 
frame ,  etc . 

d.  Consistency  checks.     Since  the  shells  can  be 
altered  by  the  novice  designers,  there  would  need  to  be  some 
consistency  checking  of  proposed  designs  to  insure  that 
building  codes  had  not  been  violated,  that  original  design 
constraints  (e.g.,  cost,  square  footage,  etc.)  had  been 
satisfied,  and  that  basic  design  principles  had  been 
followed. 

2.  In  order  to  provide  this  kind  of  advice  and  assistance, 
the  program  would  require  the  following: 

a.     A  set  of  rules  representing  existing  building 
codes. 
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b.  A  set  of  rules  representing  basic  design 
principles . 

c.  A  set  of  examples  representing  a  variety  of 
building  designs. 

d.  A  representational  scheme  which  allows  the  previous 
three  items  to  be  related. 

e.  A  mechanism  which  could  interpret  a  given  design 
goal,  select  an  appropriate  shell,  and  allow  changes  that 
did  not  violate  any  rules. 

3.  It  would  be  desirable  to  embed  this  design  advice  in  a 
program  which  automates  the  more  tedious  aspects  of  design 
(e.g.,  a  computer-aided  design  (CAD)  program). 

4.  Providing  this  type  of  design  advice  is  possible,  but  it 
is  also  complex  and  requires  a  significant  initial 
investment.     The  minimum  that  should  be  attempted  is  a  set 
of  worked  examples  along  with  the  high  level  reminders.  The 
configurable  shells  can  be  developed  one  at  a  time  starting 
with  the  most  frequently  used  shell   (e.g.,  one-story, 
single-family,  three  bedroom,  two  bath,  brick,  etc.). 


As  with  most  fantasies,  this  one  has  a  happy  ending.  This 
firm  has  gone  on  to  great  success.     They  sold  their  home  design 
advisor  program  to  Homes  'R  Us  and  purchased  the  entire  town  of 
Dime  Box  with  the  proceeds.     (P.S.     Dime  Box  has  since 
incorporated  and  been  renamed  Golden  Nibblet.) 

Instructional  Design  Guidance 

The  previous  scenario  was  introduced  because  home  designs 
result  in  familiar  physical  structures,  which  makes  it  easier  to 
imagine  what  design  expertise  is  essential  and  how  it  might  be 
represented  and  made  available  to  novices.     Instructional  designs 
result  in  plans  of  instruction  which  are  used  to  create  or  modify 
mental  structures  and  associated  behaviors  in  learners.  Mental 
structures  are  not  so  readily  accessible  for  evaluation,  which 
makes  assessment  of  instructional  designs  more  difficult. 
Nevertheless,  we  argue  that  the  situation  is  similar  to  the  home 
design  situation  sketched  above  and  that  it  is  useful  to  think  of 
automated  instructional  design  advice  in  an  analogous  fashion. 

V/e  shall  assume  that  our  novice  instructional  designer 
either  has  subject  matter  expertise  or  has  access  to  a  subject 
matter  expert  (SME) .     Furthermore,  we  shall  assume  that  our 
novice  designer  has  had  at  least  a  short  course  (perhaps  the 
equivalent  of  one  college  semester's  work)   in  instructional 
design.     We  are  now  positioned  to  consider  whether  and  what 
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instructional  design  expertise  can  be  automated  and  offered  to 
such  users. 

It  is  certainly  possible  to  offer  high  level  reminders.  In 
the  case  of  instructional  design,  these  amount  to  something  like 
Gagne's  nine  events  of  instruction  (1985).     These  events  can  be 
provided  in  a  context  sensitive  manner.     If  the  user  is  involved 
in  a  particular  event  (e.g.,  gain  attention  —  event  #1)  and  asks 
the  system  for  help,  then  the  system  would  provide  elaboration  of 
techniques  for  gaining  attention.     If  the  user  has  just  completed 
an  event  and  is  wondering  what  to  do  next,  then  the  system  could 
provide  an  overview  of  the  next  event.     If  some  scheme  other  than 
Gagne's  nine  events  of  instruction  were  used  as  an  organizing 
schema  for  the  instructional  design  advisor,  then  that  scheme 
would  have  useful  high  level  reminders  which  could  be  elaborated 
at  the  user's  request. 

To  illustrate  these  high  level  reminders,   it  is  desirable  to 
have  fairly  complete  examples  in  a  variety  of  domains  so  that  an 
example  sufficiently  close  to  the  case  at  hand  can  be  offered  to 
the  user*     These  examples  or  cases  can  be  cross-indexed  by 
instructional  objective,   instructional  event,  and  subject  domain. 
In  short,  context-sensitive  example  plans  can  also  be 
incorporated  in-+to  an  instructional  design  advisor.     In  fact, 
GAIDA  is  an  existence  proof  of  th:  possibility  of  accomplishing 
these  first  two  kinds  of  intelligent  instructional  design  advice. 

It  would  certainly  be  useful  to  provide  novice  designers 
with  the  equivalent  of  configurable  shells  —  intelligent  design 
frameworks  for  lesson  development.     Merrill  et  al.  are  attempting 
to  do  just  this  with  the  notion  of  a  transaction  shell  in  their 
second  generation  instructional  design  theory  (1991) .  An 
evaluation  of  an  early  prototype  of  an  instructional  development 
shell  for  teaching  nomenclature  (name,  location,  and  function  of 
parts  of  a  device)  clearly  indicates  that  in  some  cases  it  is 
possible  to  provide  meaningful  shells  (Specter  &  Muraida,  1991) . 

The  difficulty  or  challenge  with  regard  to  configurable 
shells  is  in  generating  sufficient  rules  and  details  to  support  a 
much  wider  variety  of  learning  objectives  and  subject  matter. 
This  is  the  challenge  addressed  by  XAIDA.     It  is  still  too  early 
to  say  to  what  extent  we  will  achieve  success  in  this  area,  but 
four  shells  designed  by  Merrill  and  colleagues  at  Utah  State 
University  and  engineered  by  Mei  Technology,  Inc.,  are  currently 
being  field  tested.     These  shells  are:     Identify,  Classify, 
Interpret,  and  Decide.     The  first  two  concern  mainly  declarative 
knowledge  involving  objects  or  entities;  the  second  two  involve 
processes  and  procedures.     The  subject  domain  chosen  for 
experimental  purposes  is  electronics  maintenance. 

The  analog  of  a  consistency  checker  is  conceivable  for  an 
instructional  design  advisor  (Duchastel,  1990) ,     Such  a  system 
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would  act  as  a  critic  and  inform  the  novice  designer  when  a  basic 
rule  or  principle  had  been  violated. 


Conclusions 

In  general  it  appears  that  some  instructional  design 
expertise  can  be  captured  in  automated  systems •     In  addition^ 
both  case-based  and  rule-based  systems  have  a  role  to  play  in 
this  enterprise.     In  short,  the  t«  '^hniques  of  artificial 
intelligence  do  have  a  legitimate  application  in  the  domain  of 
instructional  systems  design  and  development. 

Both  GAIDA  and  XAIDA  provide  the  potential  to  conduct  a 
great  deal  of  empirical  research  in  the  area  of  instructional 
design.     GAIDA  can  be  used  as  a  mechanism  for  decompiling 
instructional  design  expertise.     XAIDA  can  be  used  to 
systematically  alter  specific  instructional  parameters  in  a  shell 
(e.g.,  sequencing  of  events,  placement  of  objects  on  the  screen, 
time  allotted  for  learning  activities,  etc.)   in  order  to 
determine  optimal  default  settings  for  a  range  of  instructional 
situations  (different  learning  objectives,   learner  profiles, 
delivery  media,  etcO- 

Finally,  these  and  other  automated  instructional  design 
systems  and  tools  will  provide  the  means  for  determining  whether 
and  to  what  extent  instructional  design  is  a  science  or  art.  Our 
belief  is  that  there  are  scientific  aspects  of  instructional 
design  (e.g.,  designing  materials  so  as  to  minimize  the  load  on 
working  memory  appears  to  be  empirically  well-established) .  but 
there  are  also  artistic  aspects  of  instructional  design  (e.g., 
designing  a  specific  event  that  is  relevant  to  the  instructional 
task  at  hand  which  gains  and  is  likely  to  maintain  attention 
appears  to  be  a  task  for  a  creative  artisan. 
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CBT  the  soft  way. 


Lorna  Uden-     Staffordshire  University, 

Abstract. 

This  paper  examines  the  state  of  the  art  of  computer-based  training 
(CBT)  design  and  reviews  some  of  the  common  practises  currently 
advocated  by  various  CBT  designers.  Currently  there  is  no  standard 
"ethodology  available  for  CBT  design.  This  situation  is  akin  to 
the  software  crisis  problems  facing  computer  developers  a  decade 
ago.  Software  engineering  encompasses  a  set  of  three  key  elements 
-  methods,  tools  and  procedures  that  enable  the  developers  to 
control  the  process  of  software  development,  and  provides  the 
practitioner  with  a  foundation  for  building  high  quality  software 
in  a  productive  manner.  The  softwariB  engineering  paradigms,  the 
classic  software  development  cycle,  prototyping,  and  fourth 
generation  techniques  are  briefly  reviewed.  The  software 
development  life-cycle  with  its  different  phases  is  compared  with 
courseware  development  from  the  analysis  phase  to  design.  Several 
available  software  methodologies  and  tools  are  examined  and 
examples  show  how  Jackson  Structured  Development  and  Object- 
oriented  design  can  be  used  in  the  development  of  courseware.  The 
paper  concludes  with  a  plea  from  the  author  for  further  research  in 
this  area. 

Courseware  Instruction  Methodologies. 

Currently  there  are  no  standard  courseware  development  techniques 
or  methodologies  for  CAL  development.  Various  techniques  and 
methods  are  advocated  by  various  CAL  designers.  Despite  the  lack 
of  a  standard  methodology  for  courseware  development,  most 
proponents  of  CAL  adopt  a  form  of  systematic  approach  to  its 
design,  based  on  the  notion  of  instructional  design.  Although  the 
techniques  and  methodologies  used  vary,  the  basic  concepts  are 
similar.  An  overview  of  some  of  the  well  known  CAL  design 
methodologies  is  given  below. 

The  Three  Phase  plan. 

A  plan  is  central  to  the  production  of  a  CAL  lesson,  according  to 
this  methodology  [Steinberg  84].  The  three  phases  of  the  plan  are 
(1)  Initial  planning,  (2)  the  ripple  plan,  and  (3)  completing  the 
lesson.  The  first  and  third  phases  relate  to  the  lesson  as  a 
whole,  the  middle  phase  to  individual  units  of  instruction.  The 
three  phases  can  be  thought  of  as  planning,  producing,  and  putting 
it  together. 

The  Three  Phase  Plan  for  generating  CAL  instruction  lessons. 
I.       Initial  planning:  Lesson  level. 

Characterise  the  target  population; 

Formulate  overall  goals; 
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Analyze  the  task; 

Designate  the  pre-requisite  skills. 

II.  Ripple  plan:  Unit  level. 

Generate  the  presentation:  Program  and  evaluate; 

Expand      response      judging      and      feedback:      Program  and 

evaluate; 

Make  human  factor  and  management  decisions:  Program  and 
evaluate. 

III.  Completing  the  lesson:  Lesson  level. 
Complete  human  factor  and  management  decisions; 
Generate  introductory  displays; 

Gei  srate  concluding  displays; 

Complete  initial  set  of  evaluation  measures; 

Evaluate  and  revise; 

Document ; 

Plan  maintenance. 

A  design  model  for  courseware. 

Keller  [Keller  A.  87]  proposes  the  design  process  for  CAL  to 
consist  of  the  following  eight  steps: 

[1]     See  the  problem  in  system  terms. 

[2]     Specify  a  solution. 

[3]     Code  and  make  preliminary  tests. 

[4]     Fine-tune  for  easier  use. 

[5]     Do  a  formative  evaluation. 

[6]     Revise  as  needed. 

[ 7 ]     Implementation . 

[8]     Perform  a  summative  evaluation. 

An  eight-^step  model  for  developing  computer-based  instruction. 

The  eight  step  model  developed  by  Alessi  and  Trollip  [Alessi  & 
Trollip  85]  is  oriented  directly  toward  computer  delivery  of 
instruction.  The  model  includes  not  only  designing  a  lesson  on 
paper,  but  getting  it  into  a  computer  and  evaluating  it.  An 
important  feature  of  the  model,  claimed  by  its  proponents-  is  that 
it  contains  steps  specially  intended  to  foster  creativity  and  to 
increase  the  likelihood  that  the  developer  will  produce  lessons 
that  use  the  computer  in  new  and  more  innovative  ways. 

Many  of  the  procedures  of  the  model  are  quite  similar  to  some 
components  of  the  ISD  approach.  The  intention  is  to  take  those 
parts  of  the  ISD  approach  that  are  specifically  related  to 
individual  lesson  design,  simplify  them  for  the  beginning 
instructional  developer,  add  to  them  those  procedures  necessary  for 
delivery  by  computer,  and  incorporate  into  them  procedures  that 
will  enhance  the  creative  use  of  the  computer. 

The  model  begins  with  the  design  of  the  lesson,  starting  with  rough 


ideas,  engaging  next  in  the  creative  generation  of  ideas,  followed 
by  organising  and  analyzing  ideas.  It  then  deals  with  the 
production  of  the  lesson,  starting  with  text,  graphics,  and  other 
presentation  on  paper,  and  then  progressing  to  the  computer 
component,  the  detailed  design,  production,  and  evaluation  of  the 
computer  program. 

The  following  are  the  eight  steps: 

1.  Define  your  purpose; 

2.  Collect  resource  materials; 

3.  Generate  ideas  for  the  lesson; 

4.  Organise  your  ideas  for  the  lesson; 

5.  Produce  lesson  displays  on  paper; 

6.  Flowchart  the  lesson; 

7.  Program  the  lesson; 

8.  Evaluate  the  quality  and  effectiveness  of  the 
lesson . 

The  Roblyer  and  Hall  method • 

The  method  combines  the  Alessi  and  Trollip  method  with  the  Gagne 
design  model.  It  has  three  phases,  with  a  number  of  steps  in  each 
phase  and  several  activities  to  accomplish  at  each  step  [Roblyer 
88] •  The  proponent  of  the  method  stresses  the  importance  of  team 
work  for  effective  design.  An  optimally  effective  courseware 
design  team  consists  of  at  least  one  person  of  each  of  the 
following  types:  Instructional  design  expert,  knowledgeable  in 
courseware  design  and  learning  theory;  content  expert  who  has 
taught  in  the  area,  and  a  programmer  experienced  in  CAL 
development.  The  documentation  produced  at  each  of  the  steps  in 
the  first  two  phases  is  an  important  part  of  quality  assurance 
during  design  and  development.  This  written  information  is  the 
blueprint  for  the  product,  and  allows  everyone  on  the  design  team 
to  have  a  clear  idea  of  what  is  being  planned  before  it  is  coded^ 

Phase  I :  Design. 

Step  1.  State  instructional  goal. 

Step  2.  Perform  instructional  analysis. 

Step  3.     Develop  performance  objectives. 

Step  4.     Develop  testing  strategies. 

Step  5.     Design  instructional  strategies. 
Phase  II :  Program  development. 

Step  1.     Develop  flowchart  and  storyboards. 

Step  2.     Develop  support  materials. 

Step  3.     Review  and  revision  before  programming. 
Phase  III :  Program  development  and  evaluation. 

Step  1.     Program  first  draft  materials* 

Step  2.     Perform  formative  evaluation. 

The  system  approach  to  training  (SAT).  [R.A.F.] 

The    system    approach    to    training    (SAT) ,    sometimes    called  the 
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systematic  approach  to  training,  is  a  methodical  approach  to  the 
design  and  development  of  training  courses  used  by  the  Royal  Air 
Force  in  the  late  1960 »s  -  early  '70»s.  [RAF]  It  developed  from 
the  programmed  techniques  of  the  mid-1960 's.  SAT  is  usually 
considered  to  consist  of  seven  parts: 

[1]     Job  or  task  analysis; 

[2]     Statement  of  training  problem; 

[3]     Definition  of  objectives; 

[4]     Design  of  tests; 

[5]     Developments  of  strategies  and  methods; 
[6]     Internal  validation; 
[7]     External  validation. 

The  preceding  sections  outline  some  of  the  state-of-the-art 
methodologies  currently  used  by  CAL  designers  in  their  development 
of  courseware.  This  is  by  no  means  a  exhaustive  list.  There  seem 
to  be  many  different  methods  used,  depending  on  the  experience  and 
background  of  the  CAL  designers. 

Despite  the  different  methodologies  advocated  by  the  different 
designers,  all  of  the  methodologies  mentioned  share  many 
commonalities.  The  courseware  engineering  cycle  can  be  basically 
grouped  into  the  following  phases: 

[Ij     Requirement  analysis; 

[2]     Courseware  specification; 

[3]  Design; 

[ 4 ]     Implementation ; 

[5]     Evaluation  and  revision. 

Courseware  materials  for  CAL  are  designed  to  facilitate  learning. 
However,  much  of  the  courseware  currently  available  is  of  poor 
quality  and  fails  to  meet  standard  requirements.  Despite  there 
being  many  authoring  tools  which  proponents  claim  would  reduce 
development  costs  and  time,  little  impact  is  being  made.  The 
tools  which  are  available  are  merely  tools  for  the  implementation 
phase  of  the  cycle,  and  they  lack  guidance  for  designers  during  the 
critical  phases  of  analysis  and  design.  The  situation  is  akin  to 
the  software  crisis  many  years  ago.  Since  then  a  new  discipline 
known  as  Software  Engineering  has  emerged  which  appears  to  have 
resolved  some  of  the  problems  facing  software  developers.  The 
author  of  this  paper  believes  that  software  engineering 
methodologies  currently  available  can  be  used  to  resolve  some  of 
the  probleirs  associated  with  courseware  production.  Before 
outlining  how  this  may  be  so,  a  brief  review  of  software 
I  engineering  is  given. 

I  What  is  Software  Bnginaering? 

According  to  Fritz  Bauer  [Bauer  69],  software  engineering  is,  "the 
establishment  and  use  of  sound  engineering  principles  in  order  to 
obtain  economically  software  that  is  reliable  and  works  efficiently 
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on  real  machines". 

Software  engineering  encompasses  a  set  of  three  key  elements  - 
methods,  tools,  and  procedures  -  that  enable  the  manager  to  control 
the  process  of  software  development  and  provide  the  practitioner 
with  a  foundation  for  building  high-quality  software  in  a 
productive  manner. 

Software  engineering  is  composed  of  steps  that  encompass  the 
methods,  tools,  and  procedures.  These  steps  are  often  referred  to 
as  software  engineering  paradigms.  A  paradigm  for  software 
engineering  is  chosen  based  on  the  nature  of  the  project  and 
application,  the  methods  and  tools  to  be  used,  and  the  controls  and 
deliverables  that  are  required.     The  three  paradigms  are: 

[1]     The  classic  software  development  cycle; 

(2]  Prototyping; 

[3]    Fourth  generation  techniques. 
[1]     The  classic  software  development  cycle. 

The  figure  overleaf  shows  the  software  development  cycle,  often 
referred  to  as  the  "waterfall  model",   (see  Fig.  1.) 

(i)  Requirement  analysis.  This  stage  is  concerned  with  extracting 
from  a  customer  the  exact  requirements  that  are  needed  for  a  piece 
of  software. 

(ii)  Software  specification  is  concerned  with  dealing  with  the 
function  (purpose)  of  the  software. 

(iii)  Design  is  concerned  with  creating  descriptions  of  a  software 
system  in  terms  of  how  it  is  to  carry  out  the  functions. 

(iv)  Implementation  is  the  process  of  writing  the  programs  that 
for'a  the  software  system. 

(V)  Testing  and  Maintenance.  Testing  focuses  on  the  logical 
internals  of  the  software,  assuring  that  all  statements  have  been 
tested.  Maintenance  is  probably  the  most  important  stage  of  the 
lifecycle.  This  occurs  after  a  piece  of  software  has  been  released 
and  become  operational.  It  consists  of  modifying  the  software 
system  in  response  to  error  reports,  or  changes  in  requirements  or 
specifications . 

Quality  Assurance,  shown  to  the  right  of  the  diagram,  affects  all 
stages  of  the  lifecycle.  It  is  the  process  whereby  each  step  in 
the  development  proce^  s  is  checked,  and  has  the  aim  of  ensuring 
that  the  delivered  product  satisfies  the  requirements  identified  at 
the  start  of  the  project,  and  that  the  output  of  each  stage  meets 
the  specification  of  input  to  the  next  stage. 

The  importance  of  the  software  development  life-cycle • 
The  concept  of  software  development  life-cycle  is  important  for  two 
reasons.     First  it  recognises  and  partitions  the  act  of  developing 
software   into  different  phases,    each  phase  having  distinct  and 


385 

370 


6 


unique  activities  associated  with  it.  Second,  a  model  of  the 
softv/are  life-cycle  enables  a  software  developer  to  carry  out 
planning  activities  more  easily.  A  closer  look  at  the  software 
lifecycle  reveals  that  it  bears  a  close  resemblance  to  the 
courseware  cycle.  This  should  not  be  seen  as  surprising,  for  the 
process  of  courseware  production  is,  after  all,  the  development  of 
computer  programs  for  teaching.  In  subsequent  sections  of  this 
paper,  an  attempt  will  be  made  to  examine  how  som.e  of  the 
techniques,  methodologies  and  tools  currently  available  for 
software  engineering  could  be  used  in  the  analysis  and  design 
phases  of  courseware. 


Quality  Assurance 
is  a  feature  of 
every  phase  of 
the  life-cycle. 


Maintenance 
can  affect 
every  phase  of 
the  life-cycle, 
and  may  result 
in  changes  to 
some  or  a^,!  of 
the  documents. 


Requirement 
Analysis 


User  requirement 
Document 


Software 
Specification 


Software 
Specification 
Document 


Design 


Quality  Assurance 


Design  document 


Implementation 


Programs 


Maintenance 


Revised  programs 


Fig  !•     The  Classic  Software  Development  Lifecycle. 


The  classic  lifecycle  is  the  oldest,  and  most  widely  used  paradigm 
for    software    engineering,    however,    over    the    last    few  years. 
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criticisms  of  the  paradigm  have  caused  even  active  supporters  to 
question  its  applicability  in  all  situations*  Often  a  customer  will 
define  a  set  of  general  objectives  for  software,  but  will  not 
identify  detailed  input,  processing,  or  output  requirements.  In 
other  cases  the  developer  may  be  unsure  of  the  efficiency  of  an 
algorithm,  the  adaptability  of  an  operating  system,  or  the  form 
that  human-machine  interaction  should  take.  In  these,  and  many 
other  situations,  a  prototyping  approach  to  software  engineering 
may  be  best. 

[2]  Prototyping. 


Requirement 
gathering 


Quick 

Build 

Desian 

Prototype 

Evaluate 
and 


Engineer 
Product. 


Figure  [2].  Prototyping. 

This  is  a  process  that  enables  the  developer  to  create  a  model  of 
the  software  to  be  built,  the  model  can  take  one  of  three  forms:  a 
paper  prototype  that  depicts  human-computer  interaction  in  a  form 
that  enables  the  user  to  understand  how  such  interaction  will 
occur,  a  working  prototype  that  implements  some  subset  of  the 
functions  required  of  the  desired  software,  or  an  existing  program 
which  performs  part  or  all  of  the  function  desired,  but  has  other 
features  to  be  improved  upon  in  the  new  development  effort-  The 
sequence  of  events  for  the  prototyping  paradigm  is  illustrated  in 
figure  [2]  below. 

[3]     Fourth  generation  techniques. 

The  term  encompasses  a  broad  array  of  software  tools  that  enable 
software  developers  to  specify  some  characteristics  of  software  at 
a  higher  level.  The  tool  then  automatically  generates  source  code 
based  on  the  developer's  specification.  Currently,  a  software 
development  environment  supporting  the  4GT  paradigm  includes  some 
or  all  of  the  following  tools: 

non-procedural   languages   for   database   query,  report 
generation,    d^*ta    manipulation,    screen    interaction  and 
definition,   and  code  generation; 

high  level  graphics  capability;  and 

spreadsheet  capability. 

The  4GT  paradigm  for  software  engineering  is  depicted  in  Fig  [3] 
overleaf. 
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Requirement 
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>> 


Design 
Strategy 
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>> 


Product 


Fig  [3]-     Fourth  generation  technique. 


Implications  of  Software  Engineering  on  CAL. 

The  courseware  engineering  process,  like  software  engineering,  is 
iterative.  There  is  a  great  amount  of  feedback  from  each  phase  to 
the  previous  one. 

Engineering  techniques  for  producing  computer  software  have  been 
gaining  widespread  acceptance  recently.  This  section  examines  the 
first  three  phases  of  the  software  cycle  in  greater  detail,  and 
notes  how  the  techniques,  methodologies,  and  tools  used  in  the 
three  phases  can  also  be  utilised  for  developing  coursev/are, 

[1]     Requirement  Analysis. 

The  starting  point  of  any  software  project  is  a  document  prepared 
by  the  customer  for  a  system,  known  as  the  statement  of 
requirements.  In  it,  the  author  sets  out  in  detail  what  a  proposed 
software  system  is  required  to  do,  and  specifies  constraints  upon 
the  system  and  constraints  upon  the  development  process*  Given  a 
statement  of  requirements,  the  developer  has  to  carry  out  the 
process  of  requirement  analysis.  Requirement  analysis  consists  of 
analyzing  the  statement  of  requirements  and  extracting  and 
clarifying  the  important  parts;  i.e.  the  functions  and  the 
constraints.  The  purpose  is  to  produce  a  document  called  a  system 
sTPecif ication  or  software  specification. 

Courseware  Requirement  Analysis, 

Courseware  development  originates  with  a  customer  training  request. 
From  this  customer  training  request  the  designer  performs  the 
following  activities: 

B  identifies  the  training  goals  of  the  courseware  and  the  target 
population ; 

■  defines  how  the  training  goals  are  to  be  met; 

■  plans  the  activities  needed  to  develop  the  course. 

The  aims  of  analysis  are: 

B  to  establish  the  nature  and  content  of  the  work  for  which  the 
training  is  intended; 

n  to  determine  what  skills,  knowledge  and  attitudes  are  required 
to  perform  the  job; 

B     to  determine  what  skills,  knowledge  and  attitudes  the  student 
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already  possesses; 

n  to  document  the  training  needs  of  the  work  for  which  the 
training  is  intended. 

As  systems  analysis  is  an  activity  which  encompasses  most  of  the 
tasks  in  producing  the  user  requirements  document.  Similar  activity 
known  as  instructional  analysis  is  involved  in  identifying  the 
appropriate  skills  and  knowledge  that  should  be  involved  in 
instructional  materials  to  achieve  their  goal.  It  involves  several 
steps,   such  as: 

■  learning  task  analysis. 

■  information  processing  task  analysis, 
a  task  classification 

Learning  Task  Analysis ^  or  Hierarchical  Task  Analysis. 
Learning  task  analysis  has  been  used  for  some  time  for  designing 
instructional  models,  this  approach  helps  in  breaking  down  complex 
skills  into  more  elementary  skills  obtaining  a  hierarchy  of  the 
learners  skills  and/or  behaviours.  It  is  possible  to  obtain  a 
hierarchical  tree  which  describes  the  subdivision  of  skills  and  the 
terminal  level  corresponds  to  the  entry  level  skills  of  the 
learner. 

Au  Information-processing  Task  Analysis. 

This  approach  includes  the  follov;ing  steps:    [Merrill  •S?] 

[1]     Identify  the  operation  and  decision  steps  of  the  procedure. 

[2]       Sequence   the    steps    in   the   order    in   which   they   would  be 

performed. 

[3]     Prepare  a  flowchart  representation  of  the  sequenced  steps. 
[4]     Validate  the  flowchart  using  different  initial  inputs. 

Unfortunately^  information  processing  using  a  flowchart  to  depict 
the  sequence  of  operations  and  decision  steps  is  fraught  with 
inherent  weakness.  Recant  research  and  evaluation  of  computer 
programs  [Yourdon  &  Constantine  79]  has  shown  that  complex  programs 
using  this  methodology  suffer  significant  deficiencies.  These 
deficiencies  manifest  themselves  in  (a)  programs  that  will  not  work 
properly,  (b)  difficulty  in  finding  and  correcting  errors  in 
programs,  (c)  difficulty  in  modifying  or  expanding  programs,  and 
(d)  difficult  in  reading  or  understanding  programs. 

Computer  scientists  have  proposed  and  validated  some  new  approaches 
to  the  analysis  and  design  of  computer  programs  in  recent  years. 
By  using  one  new  approach  called  Jackson  Structured  methodology, 
each  of  the  problems  listed  above  can  be  significantly  reduced,  and 
such  an  approach  will  be  presented  later. 

Task  Classification . 

Another  step  involved  in  instructional  analysis  is  to  classify  the 
goal  statement  according  to  the  kind  of  learning  that  will  occur. 
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(Known  as  domains  of  learning)  A  number  of  ways  of  classifying  the 
capabilities'  involved  in  human  tasks  have  been  proposed.  Among 
them  are  the  categories  described  by  Bloom  and  his  co-workers 
[Bloom  65]  comprising,  Cognitive,  Effective,  and  Psychomotor;  the 
Component  Display  Theory  of  Merrill  [Merrill  83];  and  the  learning 
categories  of  Gagne  [Gagne  88]. 

The  system  of  task  classification  proposed  by  Gagne  comprises  the 
five  categories  of  verbal  information.  intellectual  skills, 
cognitive  strategy,  attitude,  and  motor  skills.  In  this  system, 
the  cognitive  domain  consists  of  the  three  major  categories  of 
verbal  information  (declarative  knowledge) ,  intellectual  knowledge 
(procedural  knowledge) ,  and  cognitive  strategy .  The  basic 
postulate  of  this  system  is  that  the  mental  processing  required  for 
learning  and  retention  of  verbal  information  is  demonstrably 
different  from  the  mental  processing  required  for  the  learning  and 
retention  of  intellectual  skills,  and  that  comparable  differences 
exist  among  the  five  categories  of  learned  capabilities.  There  are 
several  crucial  reasons  for  classifying  these  capabilities.  The 
three  reasons  given  by  Gagne  [Gagne  72]  are: 

(1)  They  can  be  used  to  provide  information  '  out 
instructional  treatments.  Research  studies  have  indicated  nat 
different  instructional  treatments  are  effective  in  promoting 
different  types  of  learning. 

(2)  To  identify  the  parts  of  various  subject  areas  that  are 
subject  to  the  same  instructional  treatments. 

(3)  Each  category  requires  different  techniques  for 
assessment.  Requirement  analysis  is  the  most  important  stage  of 
the  system  cycle  for  both  software  and  courseware  development.  The 
role  of  requirement  analysis  is  to  assist  the  designer  in  deriving 
"a  precise  and  independent  description  of  the  software  elements  of 
a  computer-based  system." 

[2]  Software  Specification* 

The  development  of  the  software  requirements  phase  focuses  on  the 
outcomes  of  the  requirement  specification  carried  out  in  phase  1  of 
the  software  cycle.  It  is  concerned  with  the  nature  and  style  of 
the  software  to  be  developed,  the  data  and  the  information  that 
T.ill  be  required  and  the  associated  structural  framework,  rhe 
required  functionality,  performance,  and  various  interfaces. 
Requirements  for  both  the  system  and  the  software  are  reviewed  for 
consistency  and  then  reviewed  by  the  user  to  be  certain  that  the 
software  requirements  faithfully  interpret  and  produce  the  system 
requirements.  A  systems  requirements  document  is  produced  in  this 
phase,  which  becomes  the  technology  and  management  guideline  which 
is  used  throughout  all  subsequent  development  phases,  including 
validation  and  testing. 

Over  the  past  decade,  a  number  of  software  analysis  methor^s  and 
techniques  have  been  developed  to  assist  designers   in  analyzing 
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software  requirements  specification.  The  methods  and  tools  can  be 
divided  into  four  broad  categories: 

B    data  flow-oriented  analysis 

B    data  structured-oriented  analysis 

B    object-oriented  analysis 

B    language  based  formal  specification. 
The  following  is  a  list  of  the  most  common  requirements-analysis 
methods  and  tools. 

D         Data     structured     system     development     -     manual  and 

automated . 

B    Information  engineering  workbench  -  automated. 

B    Jackson  System  Development  -  manual  and  automated. 

B      Software    requirement    engineering   methodology    (SREM)  - 

Formal  specification  language  and  automated. 

B  Structured  analysis  and  design  technique  -  manual  and 
automated. 


One  of  the  methodologies,  Jackson  Structured  Development  [Jackson 
83],  can  be  used  to  address  the  shortcomings  of  using  flowcharts  in 
task  analysis  mention  earlier.  When  used  in  the  context  of  JSD, 
the  structure  of  an  entity  describes  the  entity's  history  by 
considering  the  impact  of  actions  over  time.  Jackson  introduced  a 
diagrammatic  notation  known  as  Jackson  Structured  Diagram  to 
illustrate  actions  applied  to  an  entity  as  a  sequence,  as  part  of 
an  'either/or'  selection,  or  repetitively  (an  iteration). 

(1)  Sequence, 

Component  A  is  a  sequence 
of  two  components, 
B  followed  by  C. 


B 

C 

(2)  Selection. 

Box  X  is  a  selection 

or  choice  of  Y  or  Z 


condition 


(3)  Iteration. 

Box  E  is  an  iteration  of 
a  number  (possibly  zero) 
of  F's.     The  number  of 

F^s  id  controlled  by  a 
specified  condition. 
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As  mentioned  previously,  using  flowcharts  to  represent  information 
processing  is  fraught  with  inherent  weakness.  The  J.S.D. 
methodology  can  be  used  to  overcome  these  limitations.  Figure  [4] 
below  is  an  information  processing  task  analysis  using  J.S.D  to 
represent  the  procedure  for  generating  the  next  number  in  Base  3, 


Base  3 


Read  digit 
of  number. 


Process  digits 


until  no 
more  digits 


Write  1  to 
left  of  the 
last  zero. 


Digit=2  (true) 


Else 


Change 

Write 

2  to  0 

0 

Read  next 
digit  to 
left 


Increment 
digit  by  1 


Write  down 
new  numeral 


Exit 


Automated  Tools  for  Requirement  Analysis. 

The  analysis  methods  and  tools  were  originally  developed  to  be 
applied  manually.  Today  each  of  these  methods,  along  with  a  class 
of  techniques  specifically  designed  for  automated  processing,  are 
available  in  a  "computer-aided"  format.  Some  of  the  important 
automated  tools  for  requirement  analysis  are: 

(1)  SADT  -  a  structured  analysis  and  design  technique  that  has 
been  widely  used  as  a  tool  for  system  definition,  software 
requirement  analysis,   and  system/software  design  [Ross  85] . 

(2)  SREM  (Software  Requirements  Engineering  Methodology)  [Alfred 
85]    is  an  automated  requirement  analysis  tool  that  makes  use  of 
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requirements  statement  language  (RSL)  to  describe  elements, 
attributes,  relationships,  and  structures, 

(3)  PSL/PSA  (Problem  statement  language/problem  statement 
analysis)  was  developed  by  the  ISDOS  project  [Thichoew  77]  at  the 
University  of  Michigan-  PSA  provides  an  analyst  with  capabilities 
that  include: 

(i)  description  of  information  system  regardless  of  application 
area ; 

(ii)  creation  of  database  containing  descriptors  for  information 
system: 

(iii)  addition,  deletion,  and  modification  of  descriptors; 

(iv)  production  of  formatted  documentation  and  a  variety  of 
reports  on  the  specification. 

Software  Recmirement  Specification. 

The  software  requirement  specification  is  produced  at  the 
culmination  of  the  analysis  task.  The  function  and  performance 
allocated  to  software  as  part  of  system  engineering  are  refined  by 
establishing  a  complete  information  description,  a  detailed 
functional  description,  an  indication  of  performance  requirements 
and  design  constraints,  appropriate  validation  criteria,  and  other 
data  pertinent  to  requirements. 
Courseware  Specification. 

Having  completed  the  task  analysis  and  subsequently  identified  the 
requirement  specification,  the  CAL  designer  would  be  in  a  position 
to  design  the  objectives  and  criteria-test  statements  describing 
intended  instructional  outcomes,  which  are  called  objectives 
because  their  accomplishment  can  be  measured.  Course  objectives 
are  derived  from  the  analysis  requirement  specification  prepared 
earlier.  Mager  [Mager  90]  identified  three  components  of  a  well 
written  objective, 

-  the  behaviour  tc  be  exhibited 

-  the  condition  under  which  it  is  to  be  exhibited 

the   criteria   by   which   acceptable   performance   will  be 
judged. 

Following  the  requirement  analysis  performed  earlier,  the 
courseware  specification  now  contains  the  following: 

■  an  analysis  of  the  target  population  and  environment; 

topics      and      their      relevance      (according      to  defined 
classification) ; 

■  the  interconnection  between  the  different  components  of  the 
courseware; 

■  needed  prerequisites  to  enter  each  part  of  the  courseware; 

B  the  planning  of  activities,  the  employed  resources  and  time 
schedule; 

■  courseware  test  plan  which  describes  the  testing  methodology; 
m    maps  of  required  skills. 

[3]  Design. 

Software  design  is  a  process  through  which  requirements  are 
translated    into    a    representation    of    software.       System  design 
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consists  of  taking  a  functional  specification  of  a  software  system 
and  expressing  that  system  as  a  series  of  procedures.  A  set  of 
fundamental  design  concepts  has  evolved  over  the  past  three 
decades*  Each  provides  the  software  designer  with  a  foundation 
from  which  more  sophisticated  design  methodologies  can  be  applied. 

The  development  phase  must  be  instituted  regardless  of  the  software 
engineering  paradigm  applied.  Using  one  of  a  number  of  design 
methodologies,  data  design,  architectural  design,  and  procedural 
design  are  conducted.  Data  design  focuses  on  the  definition  of 
data  structures.  Architectural  design  defines  the  relationship 
among  major  structural  elements  of  the  program.  Procedural  design 
transf oirms  structural  elements  into  a  procedural  description  of  the 
software. 

There  are  a  number  of  design  methodologies  such  as  Data-flow 
Oriented  design,  Data  Structure  Oriented  design,  and  Object- 
oriented  design.  Each  software  design  method  introduces  unique 
heuristic^,  and  notation  as  well  as  a  somewhat  parochial  view  of 
what  characterises  a  design  quality.  They  all  share  a  number  of 
common  characteristics; 

(i)  A  mechanism  for  translation  of  information  domain 
representation  into  design  representation; 

(ii)  A  notation  for  representing  functional  components 
and  their  interfaces; 

(iii)  Heuristics  for  refinement  and  partitioning; 

(iv)  Guidelines  for  quality  assessment. 

There  are  a  number  of  notations  available  to  describe  the  internal 
logic  of  a  procedure.  One  such  notation  is  known  as  Design 
Structure  Diagram  (DSD)  [BSI  87]  and  shows  the  flow  of  control 
within  a  module.  In  Figure  [5],  the  DSD  describes  a  module 
accumulates  the  temperature  readings  from  a  thermocouple  taken 
every  second  over  a  minute  of  time.,  It  also  checks  that  each 
reading  is  valid  that  is,  the  reading  lies  between  the  limits  0- 
200°  C.  It  also  counts  (in  a  variable  name  total)  the  number  of 
valid  readings. 
Program  Design  Language > 

Program  design  language  (PDL)  called  structured  English,  or 
pseudocode  is  another  notation  in  detailed  design.  The 
illustration  below  shows  the  program  design  language  equivalent  of 
the  design  structure  diagram. 

MODULE  reading 

Set  total  and  count  to  zero 
REPEAT 

Input  a  reading 

Iiicrement  count 

IF  reading  is  within  0-200'''C 

THEN 


o  i  J 
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increment  total 
add  reading  to  store 
ENDIF 
UNTIL  count  is  60 
END  MODULE 


Set 

count  = 

0 

Set 

total  = 

0 

Until 
Count  =  60 


Input  a 
Reading 


Increment 
count 


Reading 

Within 

0-200''C? 


<>- 


Increment 
total 


Add  reading 
to  store 


I 


this  is  where  the  decision  is  made. 


Pig* [5]    A  Design  Structure  Diagram  (DSD) . 


Courseware  Design, 

The  aim  of  the  design  phase  is  to  provide  design  specification 
based  on  the  requirements  set  in  the  analysis  sub-phase*  The 
loilowing  steps  are  involved: 

■  Definition  of  course  modules  as  components  of  the  overall 
courseware  architecture. 
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■  Definition  of  the  overall  architecture. 

■  Educational  objectives  definition, 
o    Contents  domain  representation. 

■  Instructional  strategies  definition. 

■  Definition  of  assessment  methods. 

■I    Development  of  Standard  definition. 

■  The  script  and  storyboarding. 

The  design  of  courseware  follows  the  same  procedure  as  software 
design  by  generating  a  series  of  modules  or  procedures  for  each  of 
the  tasks  analyzed.  From  the  learning-task  analysis,  a  preliminary 
description  of  the  sequence  of  the  lesson  is  obtained.  Having 
decided  on  the  mode  of  instruction  to  be  implemented,  lesson  design 
may  now  commence.  The  design  of  an  individual  lesson  is  very  much 
the  same  as  writing  a  program  or  procedure  in  the  software 
environment.  Many  of  the  design  techniques  and  methodologies  used 
in  software  engineering  could  be  applied  here. 

Software  engineering  deals  with  problems  similar  to  those  involved 
in  courseware  production.  CASE  tools  used  in  software  engineering 
can  be  considered  to  transfer  their  methods  and  techniques  to 
courseware  design.  A  possible  tool  could  be  an  intelligent 
adviser/guide  for : 

-     the     formal     description     of     the     objectives     in  non 

ambiguous  way. 

the  identification  on  the  basis  of  the  fundamental 
characteristics  of  each  module  (objectives,  content,  tqpics, 
prerequisites) . 

The  use  of  flowcharts  is  strongly  to  be  avoided  because  of  their 
unstructured  nature.  Better  methodologies  such  as  Jackson 
Structured  Programming  (JSP) ,  Program  design  language  (PDL) ,  and 
the  use  of  design  structure  diagrams  are  strongly  recommended.  All 
of  the  above  methodologies  may  be  used  in  conducting  task  analysis. 
The  design  phase  completes  with  the  production  of  the  courseware 
design  specification  document. 

Obi ect  Oriented  Design. 

One  of  the  methodologies,  known  as  Object-Oriented  Design,  offers 
great  potential  for  courseware  design.  There  are  various  object- 
oriented  analysis  and  design  methodologies  available.  The 
advantages  of  object  orientation  over  traditional  functional 
decomposition  are  as  follows: 

■  integration  of  abstractions  viewpoints  is  possible; 

BB    higher  productivity  through  reuse  of  software  components; 

■  more  extensible  architectures  through  the  behavioural 
partitioning  provided  by  inheritance; 

■  easier  maintenance  and  system  evaluation  because  of  strong 
partitioning  resulting  from  encapsulation  and  uniform  object 
interfaces . 

Object-oriented  techniques  can  be  used  for  the  analysis  and  design 
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of  a  software  system.  Object-oriented  design  is  a  method  of  design 
encompassing  the  process  of  object  oriented  decomposition  and  a 
notation  for  depicting  both  the  logical  and  the  physical  as  well  as 
the  static  and  dynamic  models  of  the  system  under  design. 

Object-oriented  design  decomposes  a  system  into  objects,  the  basic 
components  of  the  design-  The  object-oriented  approach  attempts  to 
manage  the  complexity  it  inherits  in  the  real  world  by  abstracting 
knowledge  and  encapsulating  it  in  objects.  Finding  or  creating 
these  objects  is  a  problem  of  structuring  knowledge  and  activities. 
The  knowledge  captured  within  an  object  can  be  partly  hidden  from 
th3  external  view.  This  is  known  as  information  hiding.  The 
mechanism  for  invoking  the  use  of  an  object  is  done  by  passing 
messages  betv/een  the  objects,  which  invokes  one  of  its  operations  * 
This  operation  is  called  a  method ,  A  particular  object  is  an 
instance  of  a  class*  All  objects  of  one  kind  belong  to  a  class. 
A  benefit  results  from  the  fact  that  a  class  normally  serves  as  a 
template  to  construct  an  instance,  so  achieving  a  sort  of  reusable 
design. 

In  order  to  build  even  higher  levels  of  abstraction,  the  object- 
oriented  approach  introduces  a  concept  called  inheritance .  which 
allows  us  to  derive  more  concrete  objects  from  more  abstract  ones. 
For  example;  if  we  wish  to  express  the  fact  that  tigers  and  lions 
are  specialisations  of  cat,  we  would  first  build  a  class  'cat* 
which  describes  the  general  properties  of  cats,  then  create 
subclasses  of  'cat'  for  tigers  and  lions  which  inherit  the  general 
properties    and  can  then  be  extended  by  their  special  properties. 

The  benefit  from  this  sort  of  abstraction  is  that  objects  (classes) 
can  be  reused  and  the  amount  of  object  descriptions  significantly 
reduced • 

A  method  known  as  CRC-Method  introduced  by  Kent  Beck  and  Ward 
Cunningham  would  be  used  here.  This  method  separates  an  object 
into  its  name  (class) ,  its  behaviour  (responsibilities)  ,  and  the 
objects  it  has  relations  to  (collaboration) .  Index  cards  are  used 
to  document  an  object.  The  CRC  cards  an  be  used  for  collaborative 
design  decisions  by  a  group  of  people. 

The  CRC  Method- 


Course 

Describe  what  to  teach 

Goal 

and  how  to  teach  • . •  • 

Instruction. 
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The  top  of  the  card  contains  classname,  the  right  part  is  a  list  of 
collaborations,  and  the  lower  left  part  contains  a  description  of 
responsibilities . 

The  following  section  shows  how  the  object-oriented  technique  can 
be  used  in  courseware  development.  Based  on  the  CRC  method,  some 
of  the  objects  which  are  needed  are  presented. 

Instructional  Design  Objects  are  those  which  deal  with  the 
instructional  design  aspects  of  course  planning  and  development. 

Partial  Class  Hierarchy  of  Instructional  Design  Classes. 
Problem  Statement 
Instructional  goal 

objective 
Instructional  analysis 
Instructional  class  entry 

Theory 

Concept 

Fact 

Procedure/process 
Task 

Link 

Instructional  analysis  Ixnk 
Subordinate 

parr-of -taxonomy 
kind-of --taxonomy 
prerequisite 
reference 
Sequence 

f ollow-on-activity 
Instructional  strategy 


Associated  with  each  class  would  be  methods  to  carry  out  the 
necessary  operations  for  that  particular  class.  Messages  are  also 
specified  for  each  class. 

Approaching  the  design  using  object-oriented  method  enables  the 
designer  to  maintain  modvilarity  and  reliability,  thus  saving 
valuable  time.  Another  benefit  of  using  this  approach  is  for  rapid 
prototyping.  Object-oriented  methodology  lends  itself  to  rapid 
prototyping  because  it  contains  a  number  of  facilities 
inheritance,  polymorphism,  abstraction,  which  enable  the  character 
of  a  system  to  be  changed  quickly,  and  consequently  can  be  used  to 
develop  rapidly  a  prototype  which  can  easily  be  changed. 

Conclusion. 

The    evolution    of    applying    an    engineering     approach    to  the 
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development  of  software  has  provided  some  of  the  solutions  to 
problems  facing  software  developers,  and  offered  some  hope  of 
improved  productivity.  Today  this  approach  is  fast  becoming  the  de 
facto  standard  for  producing  software  systems.  Whereas  there  are 
techniques,  methodologies  and  tools  available  for  software 
developers  to  use  during  the  development  lifecycle,  the  situation 
is  not  so  encouraging  for  courseware  production.  However,  the 
software  engineering  discipline  does  offer  tremendous  potential  for 
courseware  developers  to  adopt  and  adapt  the  various  techniques, 
methodologies,  and  tools  to  their  own  craft.  This  paper  has  only 
managed  to  scratch  the  surface  of  the  potential  tools  available  to 
courseware  developers.  It  is  time  that  courseware  producers  took 
up  the  challenge  and  began  to  explore  the  possibilities.  Certainly 
more  research  should  be  undertaken  to  further  the  development  of 
intelligent  authoring  tools* 
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Development  of  an  Interactive  Multimedia  System  for  Use  in  Public  Access  Environments 

Jeanne  Gleason,  New  Mexico  State  University 

The  Natural  Resource  Extravaganza,  an  award  winning  interactive  multimedia  presentation  for 
use  in  public  delivery  settings,  was  developed  in  90  days  with  extremely  limited  time,  personnel, 
and  equipment.  It  operates  at  the  nation's  fifth  largest  state  fair,  and  shopping  malls  and  events 
in  the  Southwest. 

Unlike  "training"  multimedia  systems  designed  for  "captive"  learners,  multimedia  presentation 
in  busy,  noisy  environments  require  extra  attention  to  specific  design  issues.  The  presenter  will 
share  her  experience  with: 

-effective  lengths  and  designs  of  video  clips, 

-system  feedback  to  keep  users  engaged, 

-menu  designs, 

—user  options, 

-system  defaults, 

-basic  structure  of  the  information  to  be  delivered, 
-development  of  design  specification,  and 
-template  designs  for  economical  development. 

The  presenter  created  a  comprehensive  development  model,  called  ADOBE,  to  organize 
students,  professionals,  and  artists  to  economically  develop  an  effective  multimedia  system 
within  her  limited  budget.  She  will  share  hov/  her  design  templates  can  support  original  but 
parallel  development. 

This  system  was  developed  for  the  PC  environment,  using  touch  screen  input,  a  laser  disc, 
computer  graphics,  and  the  IMSATT  programming  language.  The  presenter  is  currently 
developing  additional  systems  using  Macintosh  systems  for  The  Smokey  Bear  Center,  the 
USDA,  and  the  Navajo  Indian  Tribe. 


A  MICAL  Project  Update 

Darken  K  Pigford  and  Greg  Baur,  Western  Kentucky  University 

The  Multimedia  Interactive  Computer  Aided  Learning  (MICAL)  research  project  was 
implemented  in  the  spring  of  1991  at  Western  Kentucky  University.  Its  four  phase  program  was 
presented  and  discussed  in  a  paper  given  at  the  1991  ADCIS  meeting  in  St.  Louis.  The  project 
is  now  running  and  is  undergoing  the  usual  peaks  and  valleys. 

The  purpose  of  this  presentation  is  to  bring  attendees  up-to-date  on  the  past,  present,  and  future 
of  the  project.  It  will  discuss  successes,  lessons  learned  and  future  directions  of  the  project. 
The  presenters  will  share  their  experiences  and  problem  solving  techniques  as  the  project  moves 
through  its  intermediate  phases. 
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An  Interactive  Video  Demonstration  Tutorial  of  the  Visual  Almanac 
that  Empowers  Preservice  and  Inservice  Teachers 

Charlotte  Scherer,  Bowling  Green  State  University 

For  man^'  teachers,  multimedia,  such  as  an  interactive  video  package,  is  still  an  overwhelming 
set  of  technological  resources  that  they  are  unsure  they  want  t  o  deal  with.  There  is  also,  among 
both  neophyte  preservice  and  inservice  teachers,  some  fear  that  technology  might  be  used  to 
replace  teachers  in  the  classroom,  or  lead  to  more  impersonalized  classrooms.  Thus,  it  becomes 
difficult  to  empower  teachers  with  the  technology  that  could  be  an  outstanding  resource  for 
enhancing  student  learning.  At  Bowling  Green  State  University,  we  introduce  neophytes  to  a 
rather  complex  interactive  video  disc  (IVD)  resource,  the  Visual  Almanac,  created  by  Apple 
Computer  to  demonstrate  the  potential  of  this  media  for  K-12  education.  We  developed  an 
interactive  video  stack  that  accesses  sample  segments  of  the  three  areas  of  the  Visual  Almanac 
program:  Collections,  Activities,  and  Composition  Workspace.  This  stack  does  not  require  the 
use  of  the  CD-ROM  player  and  can  be  used  with  a  20  megabyte  harddrive,  unlike  the  full 
package  which  requires  a  CD-ROM  player  or  a  40  megabyte  harddrive.  The  Visual  Almanac 
demonstration  tutorial  has  been  successfully  used  with  small  groups  of  teachers  and  as  an 
assignment  in  a  secondary  education  methods  course. 


Knowledge  and  Skills  of  Team  Membei-s  for  the  Preproduction  Stage 
of  Developing  IVD  Courseware 

Carol  Smith,  West  Virginia  Consult 

Utilizing  the  modified  Delphi  technique,  a  panel  of  experts  in  the  IVD  development  process 
identified  and  validated  knowledge  and  skills  required  by  seven  specified  roles  involved  in  the 
preproduction  stage  of  developing  level  HI  IVD  courseware.  The  panel  was  segmented 
according  to  expertise  in  roles  performed  on  IVD  development  teams  (role  specialist  group  and 
non-role  group  for  each  role  included  in  study).  The  panel  identified  several  knowledge  and 
skills  as  belonging  to  more  than  one  role.  The  roles  included  in  the  study  were:  project 
manager,  instructional  designer,  content  expert,  computer  programmer,  video  specialist,  script 
writer,  and  graphic  specialist.  The  panel  also  identified  levels  of  expertise  (basic,  intermediate, 
advanced)  for  each  knowledge  and  skill  factor. 

As  a  result  of 'his  study,  the  panel  achieved  consensus  on  144  factors:  76  knowledge  factors  and 
68  skill  factors  for  the  roles  included  in  the  study.  Differences  in  perceptions  of  levels  of 
expertise  for  each  knowledge  and  skill  factor  between  the  role  specialist  group  and  the  non-role 
group  were  determined.  One  conclusion  of  the  study  was  that  the  roles  of  video  specialist  and 
content  expert  were  the  least  understood  roles  for  the  non-role  groups. 
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Empowerment  through  Museums  and  Multimedia 

Paul  Wangemann,  Motorola  Museum  of  Electronics 

The  Motorola  Museum  of  Electronics  and  local  school  districts  in  the  Schaumburg,  Illinois  area 
have  formed  a  partnership  to  develop  and  implement  a  hands-on,  inquiry-based  approach  to 
science  teaching  and  learning.  Hands-on  classroom  activities  and  developed  curriculum  provide 
the  background  and  foundation  for  understanding  the  technologies  that  are  presented  at  the 
Motorola  Museum.  Problem-solving  activities  at  the  museum  utilize  the  science  concepts 
learned  in  the  classroom.  In  the  museum  these  concepts  are  demonstrated  through  real  world 
applications.  Group  activities  in  the  classroom  after  an  experience  at  the  museum  focus  on 
societal  applications,  explanations,  and  implications  for  technological  solutions  to  human 
problems  based  on  the  technology  studied  at  the  museum. 

The  underlying  assumption  of  the  educational  experiences  available  in  the  museum  is  that  the 
learning  process  and  environment  are  as  important  as  the  subject  matter.  The  museum  integrates 
multimedia  interactive  video  experiences  into  exhibits  and  provides  extensive  learning  activities 
in  computer  and  hands-on  laboratories.  In  order  for  visitors  to  learn  about  the  world  of 
electronics  they  need  to  experience  in  a  meaningful  way.  Computers  play  an  important  role  not 
only  as  a  tool  in  the  educational  process  but  as  a  motivating  example  of  how  electronics  helps 
to  do  important  things  differently  than  in  the  past. 

Embedding  Instructional  Delivery  Systems  for  Maximum  Effectiveness 

Stuart  //.  Weinstein  and  William  H.  Skipper,  SWL  Inc. 

This  comprehensive  session  will  examine  the  instructional  effectiveness,  time  savings,  and  cost 
benefits  resulting  from  the  use  of  high  technology  instructional  delivery  systems  that  are  coupled 
with  actual  equipment  in  the  job  setting.  The  presenters  will  guide  conference  participants 
through  the  analysis,  design,  development,  and  delivery  of  a  performance-based  bank  teller 
training  program  as  a  means  of  illustrating  the  use  of  training  technology  (computer-controlled 
interactive  videodisc)  with  actual  bank  equipment  (teller  machines)  in  the  real  job  serting  (a  bank 
branch  office). 

Based  on  the  bank  training  demonstration  and  personal  training  experiences,  conference 
participants  will  generate  an  approach  to  determine  the  appropriateness  for  training  with 
multimedia  technology  that  is  coupled  with  actual  equipment.  The  first  goal  of  this  session  will 
be  to  give  the  conference  participant  the  ability  to  identify  needs  and  processes  associated  with 
the  analysis,  design,  development  and  delivery  of  "on-demand,"  flexible,  and  cost-beneficial 
training.  Through  a  participatory  exercise  the  participants  will  then  determine  opportunities, 
benefits,  methods,  and  applications  for  the  use  of  actual  equipment  in  the  design  and 
development  of  training  for  business,  industry,  education,  military,  and  government  clients. 
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ADAPTIVE  INSTRUCTIONAL  USE  OF 
INTERACTIVE  ^/IDEODISC  PATIENT  SIMULATIONS 

Paunpimon  Amomkul, 
Lowell  A.  Schoer,  & 
Michael  W»  Fiiikelstein 

University  of  Iowa 


Abstract 

This  study  compared  the  effectiveness  and 
efficiency  of  adaptive  instruction  with  that  of  non- 
adaptive  instruction  with  an  interactive  videodisc 
patient  simulations.  Interactive  videodisc  patient 
simulations  are  used  to  teach  clinical  problem- 
solving  in  the  health  sciences.  Sixty-two 
sophomore  dental  student*',  at  the  University  of 
Iowa  College  of  Dentistry  were  subjects.  An 
Analysis  of  Covariance  (ANCOVA)  using  a  two 
by  two  factorial  design  was  utilized  to  compute 
the  obtained  data.  A  pre-test  was  set  as  a 
covariate  variable.  Two  instructional  strategies 
(adaptive  and  non-adaptive)  and  two  levels  of  pre- 
instructional  GPA  (high  and  average  identified  by 
above  versus  below  3.29  median)  were 
independent  variables. 

The  results  indicated  that  both  instructional 
strategies  were  equally  effective  in  teaching 
problem-solving,  as  measured  by  student 
achievement  and  student  retention.  However, 
adaptive  instruction  was  significantly  more 
efficient  than  non-adaptive  instruction  in  terms  of 
student  study  time.  There  was  no  interaction 
between  instructional  strategies  and  pre- 
instructional  GPA. 

Introduction 

Problem-solving  in  Dental  Education 

A  dental  curriculum  integrates  basic  science 
and  clinical  dental  sciences  using  a  variety  of 
teaching  metliods  including  lectures,  seminars, 
laboratory  practice,  clinic  practice,  and  simulated 
clinic  practice.  Even  though  dental  students  have 
equal  opportunities  to  acquire  knowledge  of  the 


basic  sciences  and  technical  skills,  differences  in 
clinical  problem-solving  skills  exist. 

The  term  "problem-solving"  used  in  this 
study  is  modified  from  McGuire's  (1985)  medical 
education  term.  It  encompasses  all  cognitive  skills 
implied  in  patient  evaluation  and  management. 
These  include  patient  information-gathering, 
diagnostic  reasoning,  medical  decision-making, 
and  patient  treatment. 

In  a  clinic,  students  diagnose  and  manage 
patients'  diseases  with  the  help  of  competent  and 
experienced  faculty.  However,  the  cases  which 
are  presented  in  the  clinic  may  lack  sufficient 
variety  to  thoroughly  develop  students'  problem- 
solving  skills.  In  addition,  limited  time  may 
restrict  students'  experiences.  Students  may, 
therefore,  lack  experience  in  developing  a 
systematic  approach  to  gathering  information  and 
solving  diagnostic  and  management  problems 
across  the  full  range  of  situations  they  are  likely 
to  encounter  in  regular  practice* 

Simulations  in  Dental  Education 

Simulations  give  practice  in  dealing  with 
some  aspect  of  the  environment  by  imitating  or 
replicating  it  (Alessi  &  Trollip,  1985).  It  ir;  an 
aid  that  brings  a  close  approximation  of  reality  to 
students.  In  tiie  traditional  curriculum,  students 
usually  deal  with  physical  simulations  of  dental 
practice  in  a  clinic  setting.  Simulations  have  not 
been  widely  used  to  teach  problem-solving 
strategies. 

The  University  of  Iowa  College  of  Dentistry 
has  developed  a  series  of  interactive  videodisc 
simulations,  called  the  Dental  Diagnosis  and 
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Treatment  (DDx&Tk)  System,  The  purpose  is  to 
teach  problem-solving  skills  by  using  an 
interactive  videodisc  system  to  deliver  a  variety  of 
dental  cases  to  students. 

The  DDx&Tx  System  uses  a  laser-reflective 
videodisc  controlled  by  a  microcomputer.  The 
student  views  two  television  screens  and 
communicates  via  the  microcomputer  keyboard. 
One  screen  displays  videodisc  images  while  the 
other  displays  text.  The  system  allocs  the  student 
to  gather  visual  and  textual  information  on  the 
patient's  disease,  offer  a  clinical  diagnosis,  and 
propose  treatment  (Finkelstein,  Johnson,  &  Lilly, 
1988). 

The  advantages  of  DDx&Tk  System  are  the 
following.  First,  it  teaches  students  how  to  think 
and  make  decisions  in  a  logical,  organized 
manner.  Second,  it  delivers  active,  individualized 
patient  simulations.  Students  are  actively  involved 
in  tlie  learning  process  and  are  required  to  make 
decisions.  Third,  it  presents  a  greater  number  and 
variety  of  cases  than  students  could  see  in  die 
clinic.  Fourth,  it  can  be  used  as  review  for 
examinations.  Finally,  it  can  be  used  in 
continuing  education  courses. 

Context  of  Problem 

DDx&Tk  System  seems  to  be  a  useful 
instructional  tool.  However,  further  research  is 
needed  to  establish  its  effectiveness  and  efficiency. 
The  amount  and  complexity  of  the  system  should 
not  only  maximize  students'  achievement  but  also 
minimize  the  amount  of  time  involved.  An 
adaptive  instructional  strategy  was  implemented  in 
this  study  to  achieve  tliese  goals. 

Adaptive  Instruction.  Tlie  term  "adaptive 
instruction"  is  used  in  this  study  as  defined  by 
Tennyson  and  Rothen  (1977;  1979),  i.e., 
strategies  which  prescribe  the  optimal  amount  of 
instruction  necessary  to  achieve  a  desired 
objective.  Adaptive  instruction  involves  highly 
developed  monitoring  systems  which  individually 


assess  and  instruct  students  (Breuer  &  Hajovy, 
1987).  It  is  directly  tied  to  student  achievement. 
During  the  learning  process,  instruction  is 
adjusted  to  match  the  level  of  student 
achievement.  Low  achievers  who  respond  poorly 
will  receive  more  instruction  oi  less  complex 
instruction.  High  achievers  receive  less  and/or 
more  complex  instruction. 

The  rationale  for  adapting  instruction  is  that 
students  should  be  confronted  with  instruction  that 
is  neither  too  much  above  or  below  their  current 
levels  of  knowledge  (Tennyson,  1988).  Low 
achievers  become  frustrated  and  confused  with 
highly  complicated  instruction,  while  high 
achievers  become  quickly  bored  with  less  complex 
instruction. 

Purpose  of  Study 

The  purpose  of  this  study  was  to  examine  the 
effectiveness  and  efficiency  of  adaptive  and  non- 
adaptive  instruction  using  the  DDx&Tx  System  as 
an  instructional  material. 

This  study  was  carried  out  to  answer  three 
research  questions. 

1.  Is  adaptive  or  non-adaptive  instruction 

more  effective? 

2.  Is  adaptive  or  non-adaptive  instruction 

mere  efficient? 

3.  Do  adaptive  and  non-adaptive  instruction 

interact  with  student  pre-instructional 
CPA  as  related  to  both  effectiveness  and 
efficiency? 

Procedures 

Subjects 

Sixty-two  sophomore  dental  students  from  die 
College  of  Dentistry  at  the  University  of  Iowa 
participated  in  the  study.  They  were  enrolled  in 
a  required,  introductory  course  in  oral  pathology. 
Subjects  were  separated  into  two  groups  based  on 
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their  CPA.  The  CPAs  were  ranked  and  classified 
as  high  or  average  depending  upon  above  or 
below  the  median  (50tli  percentile  =  3.29). 
Witliin  each  group,  students  were  randomly 
assigned  to  either  Group  1,  instruction  with 
adaptive  strategy;  or  Group  2,  instruction  with  a 
fixed  number  of  simulations  (Table  1). 

Table  1 

Basic  Research  Design  and  Sample  Size 


Instructional  Strategies 

Pre-  Adaptive  Non- Adaptive 

Instructional  

GPA  Iirmediate   Delayed     Immediate  Delayed 

Post-test  Post-test  Post-test  Post-test 


High 

16 

14 

15 

15 

Average 

15 

14 

16 

15 

it 

* 

Total 

31 

^8 

31 

30 

*  Four  students  failed  to  attend  the  delayed  post- 
test  at  the  pre- scheduled  period. 


Research  Design 

There  were  two  treatment  groups.  Group  1 
used  adaptive  instruction  (varying  number  and 
complexity  of  simulations  depending  o'. 
performance).  Group  2  used  non-adaptive 
instruction  (a  fixed  number  of  simulations 
presented  regarc'less  of  performance)  (Figure  1). 

For  Group  1,  the  maximum  number  of 
simulations  (ten)  presented  in  each  difficulty  level 
was  three,  four,  and  three  simulations  in  the  low, 
medium,  and  high  levels,  respectively.  To 
identify  the  difficulty  level,  cases  identical  to  the 
simulations  were  presented  to  a  group  of  junior 
students  during  their  clerkship  exam.  Tlie  index 
of  case  difficulty  was  pre-defined  as  a  percentage 
of  this  group  who  identified  the  case  correctly. 
The  larger  the  value  of  the  index,  the  easier  the 
case. 


The  criterion  to  pass  each  difficulty  level  was 
the  students  completing  a  simulation  with  no  less 
than  80  percent  correct  of  the  total  possible 
lesion(s)  and  with  no  more  than  one  "reevaluate" 
message  appearing  during  the  diagnosis  section. 
The  "reevaluate"  function  is  a  knowledge  of  result 
(KOR)  that  informs  a  student  of  his/her  incorrect 
response  while  tracing  the  diagnosis  decision  tree 
(Figure  2).  It  is  designed  to  guide  students  to  the 
correct  decision  by  eliminating  incorrect  options. 

All  students  began  with  two  simulations  at  the 
medium  difficulty  level.  If  students  passed  the 
medium  level,  they  advanced  to  the  high  level.  If 
they  failed  the  medium  level,  they  worked  on 
simulations  in  the  low  level.  If  students  passed 
this  low  level,  they  advanced  to  the  medium  level, 
and  then  the  high  level. 

Students'  use  of  DDx&Tk  System  was 
terminated  when  they  either  passed  the  high 
difficulty  level  or  completed  ten  simulations.  The 
minimum  number  of  simulations  that  a  student  in 
Group  1  could  study  was  three.  If  students  passed 
the  second  simulation  of  medium  difficulty  level 
and  one  high  level  simulation,  the  lesson  was 
terminated.  The  maximum  number,  in  contrast, 
was  ten:  two  medium-level  simulations,  three  low- 
level  simulations,  two  more  medium-level 
simulations,  and  three  high-level  simulations. 

For  Group  2,  the  simulations  were  randomly 
selected  from  each  difficulty-level  pool.  They 
were  sequentially  presented  to  the  students  in 
levels  of  increasing  difficulty.  Each  started  with 
three  low-level  simulations,  then  proceeded  to 
four  medium-level  simulations,  and  then  to  three 
high-level  simulations. 

The  possible  maximum  number  of  simulations 
presented  to  students  in  Group  1  was  equal  to  the 
total  number  of  simulations  presented  to  students 
in  Group  2  (ten  simulations).  However,  the 
Group  1  students  could  receive  a  smaller  number 
of  simulations  depending  on  their  performance. 
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Group    1 .  Adap'bivQ 


Present  a  case 


Low  difficulty  level 


Present  a  case 


{     Present  a  case 


1  •  3  case(s)        Medium  difficulty  level 


2- 4  cases 


Present 


High  difficulty  level 


Present  a  case 


Low  difficulty  level 


Group    2.  Kon-adaptive 


Present  a  case 


Present  a  cast 


3  cases        Medium  difficulty  level 


4  cases         High  difficulty  level 


Figure  1 .  Adaptive  and  non-adaptive  Instructional  Strategies 
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Figure  2.  A  Section  of  the  Diagnosis  Decision  Tree 
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Instruments 

Pre-test  and  Post-tests.  Parallel  forms  of  a 
test  were  written  by  the  director  of  the  oral 
pathology  course.  Both  instruments  were  proven 
to  be  very  reliable  (.80  and  .82).  The  first  form 
was  used  throughout  the  data  collection  period  as 
pre-test  and  immediate  post-test.  The  other 
parallel  form  of  the  test  was  used  as  a  one-week 
delayed  post-test. 

Each  form  of  the  test  had  two  sections: 
content  and  case  analysis.  The  content  section 
consisted  of  ten  multiple  true-false  questions. 
Each  question  had  nine  choices.  Students 
identified  the  correctness  (true  or  false)  of  each 
choice.  Each  correct  choice  was  counted  as  one 
point.  The  potential  score  of  this  test  section  was 
90  points. 

The  case  analysis  section  was  constructed  to 
examine  problem-solving  skills.  This  test 
paralleled  the  DDx&Tx  System,  It  contained  five 
cases  presented  as  slides  and  descriptive 
paragraphs.  Students  studied  each  case  by  reading 
paragraphs  describing  patient  demographic 
information  and  patient  problems  and  by  viewing 
related  visuals.  Students  then  named  the  correct 
lesion(s).  The  tree  diagram  of  diagnosis,  which  is 
a  hierarchical  organization  of  lesions,  was 
provided  during  the  test.  The  total  number  of 
correct  lesions  in  these  five  cases  was  eighteen. 

Student  Study  Time.  The  computer  tracked 
the  time  each  student  took  to  complete  all 
simulations.  The  time  started  immediately  after 
the  student  finished  typing  his/her  ID  number.  It 
ended  when  students  completed  ten  simulations  or 
passed  the  highest  difficulty  level. 

Patient  diagnosis  is  time-consuming  and 
requires  the  use  of  complicated  problem-solving 
skills.  Presenting  the  maximum  of  ten  simulations 
at  one  time  could  cause  fatigue.  In  this  study, 
students  were,  therefore,  allowed  to  stop  and 
return  during  their  three  scheduled  periods  to 


complete  the  simulations.  The  entire  experiment 
lasted  twelve  weeks. 

Data  Collection 

During  the  first  period,  students  took  the  pre- 
test immediately  before  beginning  the  simulations. 
In  Group  1,  the  computer  selected  a  number  of 
simulations  based  on  students'  performance  during 
the  lesson.  A  maximum  of  four  simulations  was 
presented  to  students.  In  Group  2,  the  students 
studied  four  simulations.  The  maximum  of  four 
simulations  was  defined  for  this  period  because 
students  needed  time  to  become  acquainted  with 
the  system  and  the  simulations.  The  students 
studied  at  their  own  pace  on  their  own  time  in 
both  groups. 

During  the  second  period  trie  students 
completed  the  simulations.  In  Group  1,  the 
computer  selected  the  difficulty  level  of  the 
simulations  based  on  students'  first-session 
performance.  A  maximum  of  six  addition?] 
simulations  was  presented.  In  Group  2,  all 
students  studied  three  medium-level  and  three 
high-leve'  simulations.  Immediately  after 
completing  the  simulations,  both  groups  of 
subjects  took  the  post-test. 

A  week  later,  students  took  a  delayed  post- 
test.  A  potential  of  20  extra  points  (based  on  tlieir 
performance)  on  delayed  post-test  scores  was 
given  to  all  students. 

Results 

Since  the  content  scores  and  case  analysis 
scores  have  different  scales  of  measurement,  these 
scores  had  to  be  transformed  into  a  common 
scale.  To  do  this,  each  of  62  raw  scores  of 
content  and  of  case  analysis  was  converted  to  a 
standard  score  (z-score).  The  z-scores  of  content 
and  case  analysis  were  combined  and  transformed 
into  T-scores. 
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Effectiveness 

The  effectiveness  of  the  interactive  videodisc 
simulation  lessons  was  measured  by  students' 
post-test  scores  (immediate  and  delayed  post-test). 
The  immediate  post-test  measured  student 
achievement;  the  delayed  post-test  measured 
student  retention.  These  tests  were  evaluated  by 
an  analysis  of  covariance  (ANCOVA)  in  order  to 
examine  the  main  effect  and  interaction  between 
instructional  strategies  and  pre-instructional  GPA. 

The  immediate  post-test  scores  were  not 
significant  for  the  main  effects  of  either 
instructional  strategy  or  pre-instructional  GPA 
(p>.05).  The  interaction  effect  between 
strategies  and  GPA  was  also  not  significant 
(p>  .05)  in  any  of  the  analyses  (Table  2). 

The  only  significant  effect  found  on  the 
delayed  post-test  was  for  GPA  groups.  High 
achievers  performed  significantly  better  than 
average  achievers  (Table  2). 


Table  2 

Analysis     of     Covariance     on     Immediate  Post- 
test  and  on  One-week  Delayed  Post- test 


Source  of     Sum  of 
Variation  Squares 
(SS) 

Degrees  of  Mean 
Freedom  Square 
(df)  (MS) 

F 

P 

IMMEDIATE  POST -TEST 

Pre-test  1889.434 
strategy  0.534 
GPA  Groups  31.019 
Interaction  0.011 

1 
1 
1 
1 

1889.434 
0.534 
31.019 
0.011 

71.849 
0.020 
1.180 
0.000 

0.000 
0.387 
0.282 
0.983 

Error  1498.936 

57 

26.297 

ONE -WEEK  DELAYED  POST -TEST 

Pre-test  939.650 
Strategy  26.285 
GPA  Groups  265.223 
Interaction  2.925 

1 
1 
1 
1 

939.650 
26.285 
265.223 
2.925 

22.037 
0.616 
6.220 
0.069 

0.000 
0.436 
0.016* 
0.794 

Error  2259.871 

53 

42.639 

*  p  <  .05 


Effidency 

The  time  to  complete  the  simulations  in  the 
four  groups  is  shown  in  Table  3.  Efficiency  was 
defined  as  the  ratio  of  study  time  to  performance 
on  the  dependent  measures.  In  every  case,  the 
adaptive  conditions  is  significantly  higher  than 
non-adaptive  conditions  (Table  4).  However,  the 
ANCOVA  produced  no  significant  differences  for 
either  instructional  strategies  or  interaction. 

Table  3 

Time  (in  Minutes)  to  Complete  Simulations 


Pre-  Instructional  Strategies 

Instructional  Total 

GPA  Adapt  i  ve         Non-Adapt  i  ve 


High 

HA 

HNA 

RM 

96 

133 

115 

SD 

36-66 

9.81 

43.23 

R 

159-40 

218-61 

188-50 

N 

16 

15 

31 

Average 

AA 

ANA 

RM 

102 

125 

113 

SD 

42.79 

43.92 

43.35 

R 

220-51 

189-62 

204-56 

H 

15 

16 

31 

Total 

RH 

99 

129 

114 

SD 

39.72 

46.86 

43.29 

R 

189-45 

203-61 

196-53 

N 

31 

31 

62 

HA  -  High  achievers  in  Adaptive  instruction 

AA  -  A\.'erage  achievers  in  Adaptive  instruction 

HNA  -  High  achievers  in  Non-Adapt ive  instruction 

ANA  -  Average  achievers  in  Non-Adapt ive  instruction 

RM  -  Raw  Mean 

SD  -  standard  Deviation 

R  -  Range  (highest  score  •  lowest  score) 

N  -  Number  of  subjects 

Discussion 

Effectiveness 

The  ANCOVA  produced  non-significant 
differences  for  instructional  strategies,  pre- 
instructional  GPA,  and  their  interaction  (p>,05) 
on  the  immediate  post-test. 
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Table  4 

Analysis  of  Covariance  on  Ratio  of  Inimediate  Post- 
test  scores  to  Study  Time  and  on  Ratio  of  One- 
week  Delayed  Post -test  Scores  to  Study  Time 


Source  of     Sum  of    Degrees  of   Mean         F  p 
Variation     Squares    Freedom  Square 
(SS)        (df)  (MS) 


IMMEDIATE  POST-TEST/STUDY  TIME 


Pre-test 

1.146 

1 

1,146 

25.873 

0.000 

Strategy 

0.463 

1 

0.463 

10.448 

0.002* 

CPA  Groups 

0.029 

1 

0.029 

0.644 

0.426 

Interaction 

0.019 

1 

0.019 

0.419 

0.520 

Error 

2.525 

57 

0.044 

ONE -WEEK  DELAYED  POST -TEST/STUDY  TIME 


Pre-test 

1.024 

1 

1.024 

24.563 

0.000 

Strategy 

0.545 

1 

0.545 

13.066 

0.001* 

CPA  Groups 

0.011 

1 

0.011 

0.257 

0.614 

Interaction 

0.000 

1 

0.000 

0.004 

0.950 

Error 

2,210 

53 

0.042 

*  p  <  .05 

The  ANCOVA  results  of  the  two  post-test 
scores  were  used  to  answer  the  research  questions: 
"Is  adaptive  or  non-adaptive  instruction  more 
effective?"  and  "Do  adaptive  and  non-adaptive 
instructions  Interact  with  student's  pre- 
instructional  GPA  related  to  effectiveness?"  The 
answer  to  both  questions  is  "No",  at  least  within 
the  limit  of  the  present  study. 

The  students  used  in  this  study  seemed  to 
learn  as  much  from  non-adaptive  as  from  adaptive 
instruction  and  this  effect  did  not  interact  with 
GPA.  Although  GPA  did  have  a  significant  effect 
on  delayed  post-test  performance,  the  effect  was 
not  significantly  different  for  the  tv/o  strategies 
used  in  this  study. 

Efficiency 

The  efficiency  of  adaptive  instruction  and 
non-adaptive  instruction  was  a  major  concern  of 


this  study.  The  ANCOVA  produced  significant 
differences  for  instructional  strategies  on  both 
student  achievement  and  student  retention. 

lliese  data  provide  the  answer  to  the  research 
question,  "Is  adaptive  or  non-adaptive  instruction 
more  efficient?".  The  results  of  this  study  suggest 
that  adaptive  instruction  is  more  efficient  than 
non-adaptive  instruction. 

Students  in  the  adaptive  group,  on  average, 
received  fewer  simulations  than  students  in  the 
non-adaptive  group.  This  smaller  number  of 
simulations  they  received  reduced  the  amount  of 
time  they  spent.  Interestingly,  less  time  spent  on 
instruction  did  not  create  any  disadvantage  in  the 
learning  or  retention. 

Limitations  of  the  Study 

Two  major  limitations  in  this  study  are  the 
content  used.  More  content  areas  in  dental 
education  could  have  been  used,  but  resource 
limitations  would  permit  only  one. 

Another  limitation  was  that  students  had 
different  levels  of  computer  proficiency.  Prior  to 
this  study  most  of  the  students  had  little  or  no 
direct  experience  in  operating  computers.  To 
reduce  this  effect,  orientation  sessions  were 
provided  to  introduce  students  to  the  interactive 
videodisc  system  and  to  demonstrate  how  to 
operate  the  patient  simulations.  Students  who  had 
the  least  computer  experience  were  encouraged  to 
operate  the  machine  during  the  orientation. 
However,  orientation  was  only  an  introduction  to 
the  system.  Some  students  needed  more  advice 
and  practice  to  operate  the  system  than  others. 
During  the  actual  simulations,  time  spent  on 
advice  and  practice  was  counted  as  part  of  the 
student  study  time. 

Implications  for  Practice 

Adaptive  simulations  proved  to  be  effective 
and  to  decrease  student  study  time.  It  also  saves 
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faculty  time  in  judging  and  prescribing  the 
appropriate  amount  and  difficulty  levels  of 
practice  each  student  neeus  in  order  to  diagnose 
patients  effectively  and  efficiently.  Further,  it  can 
help  advise  or  answer  individual  student  questions 
by  inserting  results,  discussion,  and  treatment 
plans  related  to  specific  questions  into  the 
simulations. 

Most  computerized  instruction  is 
individualized  instruction.  The  main  purpose  of 
developing  this  instruction  is  to  allow  students  to 
study  a  predefined  lesson  at  tlieir  own  pace. 
Advanced  lessons  may  be  provided  after  they  have 
finished  some  specific  lessons.  That  was  the 
situation  in  this  study.  This  instruction  not  only 
provided  a  self-paced  strategy,  but  also  provided 
each  student  distinctive  and  effective  instruction. 
In  this  study,  students  received  different  numbers 
of  simulations  at  different  levels  of  difficulty 
based  on  their  performance. 
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Interactive  Audio:    Research  and  Practice 

Dr.  Ann  Barron,  University  of  South  Florida 


ABSTRACT 

Interactive  audio  is  an  increasingly  imponant  component  of  multimedia  instruction*  Instructional 
designers  and  developers  are  faced  with  many  alternatives  in  hardware,  software,  implementation 
techniques,  and  design  strategies.  This  paper  focuses  on  appropriate  applications  for  incorporating 
audio  into  interactive  lessons  and  the  advantages  and  disadvantages  of  using  digital  audio, 

INTRODUCTION 

Interactive  courseware  has  been  a  largely  text-based  medium  for  many  years.  When  audio  was  first 
introduced  into  CBT,  it  was  utilized  primarily  as  feedback  for  inappropriate  input  by  students.  As 
software  and  hardware  improved,  audio  in  the  form  of  music  was  added  for  reinforcement  of  correct 
answers  and  for  transitions  between  lesson  segments. 

With  the  advancement  of  interactive  videodisc  technology  in  the  1970s,  audio  became  a  viable 
option.  In  this  format,  audio  was  used  to  provide  spoken  instructions,  include  sound  effects,  create 
scenarios,  explain  concepts,  and  prescribe  remediation  (Pratt  &  Trainor,  1990), 

The  introduction  of  digital  audio  in  1982  created  circumstances  necessary  for  effective  integration  of 
synchronized,  random  audio  into  CBT.  Modem  digital  audio  computer  cards  enable  CBT  developers  to 
record,  store,  edit,  and  play  back  segments  of  audio.  Control  of  these  segments  tlirough  computer 
software  programs  is  precise  and  reliable. 

An  abundance  of  digital  audio  cards  and  peripherals  is  now  appearing  on  the  market.  The  chart  in 
Figure  1  represents  common  digital  audio  boards  utilized  in  computer-based  training. 


;  iAOpii&CAf=^^ 

.     ..COMPA^}V   •  : 

■  PRICE 

DigiSpeech 

DigiSpeech  Inc. 

$198.00 

1.3.4.8 

DSA-620 

Online  Computer 
Systems 

$1441.00 

4,6,8,12.16.18.37.8 

M- Audio  Capture 
&  Playback 

IBM 

$370.00 

8.11.22,44 

MicroKey  Audio 
Card 

Video  Associates 

$495.00 
$295.00 

8.16,32 

ProAudio  Spectrum 

Media  Vision 

$398.95 

6,8,11,22.32.44 

SoundBlaster  Pro 

Creative  Labs 

$299.95 

4-44 

SoundMasler  II 

Covox 

$236.95 

4.6-25.4 

VP625 

Antex 

$450.00 

8.12,16 

Figure  1.  Digital  Audio  Boards 
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Many  companies  are  also  distributing  external  digital  audio  peripherals.  For  example,  the 
AudioPort  by  Antex  can  record  and  play  audio  files  at  various  sampling  rates.  Digital  audio  cards  and 
peripherals,  combined  with  an  increase  in  computer  storage  space,  have  resulted  in  a  di  amatic  increase 
in  the  use  of  audio  in  multimedia  instruction. 


THE  RESEARCH  BASE  FOR  INCORPORATING  AUDIO 

There  are  many  appropriate  applications  for  audio  in  interactive  courseware.  Young  learners  and 
non-readers  are  natural  targets  for  instruction  in  reading(Reitsma,  1988;  Wise,  1990).  Researchers 
have  demonstrated  success  with  audio-based  interactive  courseware  for  mildly  handicapped  learners 
(Wiener,  1991).  Audio-based  instruction  is  also  appropriate  for  training  that  involves  recognition  of 
specific  sound  cues,  such  as  sonar  operations  in  the  U.S.  Navy  (Barron  &  Varnadoe,  1992). 

An  underlying  a.  ^imption,  when  adding  an  additional  channel  of  media  to  transmit  a  message,  has 
often  been  that  utilizi  on  of  additional  channels  will  effectively  increase  the  amount  of  communication 
and  the  amount  of  learning.  Reviews  of  research  in  this  area,  however,  indicated  that  studies  in 
multichannel  communication  resulted  in  "mixed  and  contradictory  findings"  (Severin,  1967a,  p.  233). 
Reasons  cited  for  the  controversy  included  poor  sampling  techniques,  weak  designs,  lack  of 
randomization,  poor  controls,  lack  of  probability  statements,  and  test  channel  bias. 

A  study  was  conducted  at  the  University  of  Central  Florida  to  analyze  the  achievement  differential 
between  various  combinations  of  audio  and  text  in  a  CBT  tutorial.  The  treatments  consisted  of  the 
delivery  of  an  identical  CBT  tutorial  through  one  of  three  modes:  (1)  text-only,  (2)  partial  text  and  full- 
audio,  and  (3)  full-text  and  full  audio.  Results  of  this  study  indicated  that  the  various  CBT  delivery 
modes  did  not  have  a  significant  effect  on  overall  comprehension  of  the  tutorial  content  (Ban'on,  1991). 

Although  this  study  concluded  that  the  addition  of  audio  did  not  increase  learning,  it  also  indicated 
that  learning  did  not  decrease  when  a  substantial  amount  of  the  textual  information  was  deleted  firom  the 
computer  screen  and  replaced  with  audio.  This  finding  can  counter  concerns  that  some  adults  require 
the  visual  input  to  succeed.  Lessons  with  complex  screens  may  be  able  to  provide  instruction  through 
the  audio  channel  without  adversely  impacting  the  lesson  effectiveness. 


ADVANTAGES  OF  DIGITAL  AUDIO 

In  the  past,  when  the  decision  was  made  to  include  audio  in  interactive  lessons,  the  primary  medium 
was  videodisc.  Audio  on  a  CAV  videodisc  is  usually  analog  and  is  limited  to  60  minutes  per  side,  if 
both  tracks  are  fully  utilized.  There  are  several  advantages  to  digitizing  audio  and  storing  it  on  a 
computer  drive  instead  of  mastering  it  onto  a  videodisc: 

•  Ease  of  recording  and  editing.  Most  of  the  audio  boards  and  peripherals  have  accompanying 
software  for  recording  and  editing  sound  files.  To  record  audio,  a  sampling  rate  is  selected  in  the 
software  and  the  "record"  option  is  chosen.  In  many  cases,  the  software  also  includes  a  sound 
editor.  This  feature  allows  developers  to  view  the  sound  waves  and  "cut  and  paste"  the  audio. 

•  Random  access.  When  audio  is  stored  in  digital  files,  it  can  be  accessed  at  random  by  filename. 
If  an  authoring  system,  such  as  TenCORE,  is  used  for  development,  it  can  play  a  recorded  file  with 
a  DOS  call. 

•  Flexibility  for  revisions.  Digital  audio  is  stored  by  filename.  If  revisions  are  required,  developers 
can  merely  record  the  file  again  and  replace  the  previous  recording  by  using  the  same  filename. 
With  this  method,  changes  to  the  authoring  code  will  not  be  required  to  revise  the  audio. 
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•  Not  tied  to  video.  Audio  on  a  videodisc  in  generally  stored  in  analog  form  and  is  tied  to  the  video 
content.  When  audio  is  stored  in  digital  files,  it  cai^  be  played  independent  of  the  video  on  a 
videcxlisc.  In  other  words,  the  audio  is  no  longer  tied  to  the  video  component.  This  feature  makes 
it  possible  lo  include  audio  in  conjunction  with  still  ft-ames  and  various  speeds  of  video. 

•  Capacity  limited  only  by  storage.  Audio  on  a  videodisc  is  limited  to  60  minutes  per  videodisc 
side,  if  both  audio  tracks  are  fully  utilized.  When  audio  is  stored  in  digital  form,  the  capacity  is 
limited  only  by  the  available  storage.  In  many  cases,  digital  audio  is  stored  on  removable  hard 
drives  of  over  300  megabytes.  Digital  audio  can  also  be  stored  and  played  from  a  compact  disc. 

•  In-house  recording  of  classified  information.  If  classified  material  is  stored  on  a  videodisc, 
special  precautions  must  be  taken  at  the  recording  studio.  The  mastering  costs  of  classified 
videodiscs  are  often  twice  as  much  as  unclassified  videodiscs.  To  record  classified  digital  audio, 
the  same  precautions  must  be  taken;  however,  many  CBT  development  companies  have  classified 
in-house  audio  studios. 


DISADVANTAGES  OF  DIGITAL  AUDIO 

Digitizing  audio  has  several  distinct  limitations.  The  decision  to  invest  in  this  technology  must  be 
carefully  analyzed.  Following  are  some  of  the  disadvantages  of  using  digital  audio  in  courseware: 

•  Lack  of  standardized  hardware.  Most  of  the  audio  boards  in  Figure  1  are  incompatible  with  each 
other.  In  other  words,  if  you  have  an  Antex  card  in  the  development  station,  the  files  will  not  play 
in  a  delivery  station  that  has  a  SoundBlaster  Pro.  Two  trends  may  help  to  alleviate  some  of  the 
incompatibility  problems  in  the  future.  First,  the  multimedia  extensions  of  Microsoft  Windows  are 
simplifying  the  process  of  loading  drivers  for  various  sound  boards.  Secondly,  audio  peripherals, 
such  as  the  AudioPort,  make  it  easier  to  connect  and  transport  sound  digitizing  hardware. 

•  Requires  large  amount  of  RAM.  Some  of  the  audio  digitizing  techniques  require  a  large  amount 
of  RAM  because  the  files  must  be  loaded  into  RAM  before  they  are  played.  In  addition,  the  drivers 
for  the  audio  device  must  be  resident  in  RAM.  When  purchasing  an  audio  card,  check  to  determine 
whether  it  records  to  RAM,  to  disk,  or  provides  both  options. 

•  Requires  large  amount  of  disk  storage  space.  Audio  files  are  extremely  memory  intensive. 
Figure  2  shows  the  requirements  for  storing  audio  at  various  sampling  rates.  Compression 
techniques  are  being  perfected  that  may  help  to  constrain  the  amount  of  disk  space  required  for 
audio  files  in  the  future. 


SAMPLING  RATE 

STORAGE  FOR  1 
SECOND  OF  SOUND 

SECONDS  OF  SOUND 
PER  1  MEGABYTE 

22  KHz 

22  Kbytes 

45  seconds 

11  KHz 

1 1  Kbytes 

90  seconds 

7  KHz 

7  Kbytes 

1 35  seconds 

5  KHz 

5  Kbytes 

1 80  seconds 

Figure  2,  Sampling  Rates  and  Storage 
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•  Quality  is  limited  by  storage  requirements.  Because  tlie  file  sizes  are  large,  developers  are  forced 
to  accept  smaller  sampling  rates  and,  therefore,  less  audio  quality  on  playback.  In  a  recent  survey, 
the  most  prevalent  sampling  rates  for  industrial  training  were  found  to  be  between  8  and  12  KHz 
(Barron,  1991). 

*  Difficult  to  synchronize.  Because  the  audio  is  not  "tied"  to  the  video  of  a  videodisc,  the^e  is  no 
inherent  synchronization.  In  addition,  if  the  audio  files  are  loaded  into  RAM  before  playing,  the 
access  time  will  vary  based  on  the  file  size,  making  synchronization  difficult, 

CONCLUSION 

Digital  audio  is  now  a  viable  option  for  interactive  coursewaie.  Appropriate  applications  for  audio 
instruction  include  reading  education,  recognition  of  specific  sound  cues,  and  replacement  for  text  on 
complex  screen  displays.  If  audio  is  chosen  as  a  delivery  medium,  digital  audio  is  advantageous  if  the 
design  requires  flexibility,  revisions,  random  access,  and  large  quantities  of  audio.  Disadvantages  of 
using  digital  audio  include  the  lack  of  standardized  hardware,  large  RAM  and  storage  requirements,  and 
reduced  quality  based  on  storage  capacity. 
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DIGITAL  AUDIO  CARDS 


PigiSpggch 
DigiSpeech,  Inc. 
1043  N.  Shoreline  Blvd. 
Mountain  View  CA  94043 
(415)  494-8086 

PSA-6?0 

Online  Computer  Systems 
20251  Century  Blvd. 
Gerniantown  MD  20874 
(800)  922-9204 

N.-^uuio  Capture  and  Playback 

IBM  United  States 

Department  7EY 

4111  Northside  Parkv/ay 

H04L1 

Atlanta  GA  30327 
(800)  772-2227 

Microkev  Audio  Card 
Video  Associates  Labs 
4926  Spicewood  Springs  Road 
Austin  TX  78759 
(512)346-5781 


Pro  Audio  Spectrum 
Media  Vision 
47221  Fremont  Blvd. 
Fremont  CA  94538 


SoundBlaster  Pro 
Creative  Labs,  Inc. 
2050  Duane  Ave. 
Santa  Clara  CA  95054 
(408)  986-1461 

SoundMaster  II 
COVOX,  Inc. 
675  Conger  St. 
Eugene  OR  97402 
(503)  342-1271 


VP625 
i\ntcx 

16100  South  Figueroa  St. 
Gaidena  CA  90248 
(213)  352-3092 
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Partners  in  Interactive  Video: 
Designing  for  Research  and  Implementation 
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Abstract 

The  PETAI  Training  Tutorial  and  Database  of 
Pedagogical  Knowledge  in  Physical  Education 
(PETAI-IV)  is  a  multifaceted  computer-assisted 
interactive  video  program  designed  and  developed 
as  a  collaborative  effort  between  the  School  of 
Kinesiology  &  Physical  Education  and  the 
Department  of  Educational  Technology  at  the 
University  of  Northern  Colorado.  In  light  of  the 
unique  aspects  of  this  collaboration,  an  analysis  of 
the  design,  development,  formative  evaluation, 
research  findings,  and  implementation  of  this  CAIV 
program  provides  an  important  source  of  inform  jl'on 
for  interactive  project  design  teams. 

Introduction 

Interactive  videodisc  instruction,  since  its  introduction 
in  1977,  has  had  relatively  limited  but  enthusiastic 
application  in  education  (Chen,  1990).  Physical 
educators  in  various  arenas  have  recentiy  begun  to 
probe  the  use  of  interactive  videodisc  environments. 
Accordingly,  interactive  videodisc  (IV)  instruction  is 
now  being  tested,  applied,  and  researched  in  the 
physical  education  setting  as  suggested  by  Gusthart 
and  Sprigings  (1989).  IV  use  in  physical  education 
includes  the  instruction  in  skill  analysis,  performance 
skills,  self-analysis,  officiating  skills,  and  teaching 
skills  (Mathias  &  Smith,  1991).  In  fact  several 
researchers  have  investigated  IV  applications  in 
physical  education  related  areas  (Horton,  1992; 
Lyness,  1985;  Mathias,  1990;  O'Sullivan,  Sti-oot, 
Tannehill,  &  Chou,  1989;  Walkley  &  Kelly,  1989). 

Owing  to  the  two  primary  benefits  of  computer- 
assisted  interactive  video-access  to  a  broader  base 
of  materials  and  the  opportunity  for  individualized 


instruction  (Pavlonnis,  1988;  Schwier,  1987;  Seal- 
Wanner,  1988),  interactive  video-based  instruction 
may  become  one  of  the  most  revolutionary 
developments  in  the  history  of  education  and  training 
(Gindele  and  Gindele,  1984).  Brody  (1984) 
recommended  a  systematic  analysis  of  interactive 
video  focusing  on  research  with  IV  and  research 
about  IV.  Gindele  and  Gindele  (1984)  emphasized 
that  educators  should  make  every  effort  to 
understand  and  to  utilize  ttiese  technologies  in  order 
to  better  manage  the  tremendous  amount  of  new 
information  that  is  constantly  evolving.  A  depiction  of 
what  has  been  ascertained  about  the  design  and 
development  of  a  physical  education  video 
instructional  program,  in  terms  of  research  and 
implementation,  will  hopefully  stimulate  future 
developments  of  this  new  educational  medium. 

Overview 

Over  the  last  three  years,  the  physical  education  and 
educational  technology  programs  at  the  University  of 
Northern  Colorado  have  developed  a  close 
collaboration  with  the  goal  of  substantially  changing 
and  improving  the  process  of  training  physical 
education  teachers.  This  collaboration  has 
developed  to  meet  several  needf^  in  both  programs. 

School  of  Kinesiology  &  Physical  Education 

The  issues  for  physical  education  center  on 
increased  course  demands  and  lack  of  student  time 
to  complete  the  courses.  Because  of  the  K-12  scope 
of  physical  educator  training,  students  must  show 
competency  in  a  wide  range  of  skills  and  teaching 
strategies.  The  physical  education  preservice 
teacher  must  show  competency  in  analysis  of  16 
basic  motor  skills,  provide  appropriate  feedback,  and 
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prescribe  interventions  appropriate  to  overcome  a 
deficiency.  Another  critical  area  is  teaching  physical 
educators  to  be  critics  of  their  own  teaching.  The 
physical  education  program  uses  a  computerized 
version  of  the  Physical  Education  Teaching 
Assessment  Instrument  (Phillips,  Cariisle,  Steffen,  & 
Stroot  1986)  to  observe  preteachers,  analyze  the 
amount  of  time  spent  on  various  teaching  and 
management  tasks,  and  provide  feedback  on  their 
performance,  in  both  cases,  basic  skills  analysis  and 
teaching  performance  analysis,  the  faculty  find  that 
there  is  not  sufficient  time  in  the  students  course  of 
study  to  master  these  important  but  diver5;e  skill 
domains. 

Department  of  Educational  Technology 

Interactive  technologies  is  one  of  three  areas  of 
specialization  within  the  newly  formed  Ph.D.  program 
offered  by  the  Department  of  Educational 
Technology.  An  important  part  of  the  training  for 
doctoral  students  in  this  area  of  specialization  is 
participation  in  the  design  and  development  of  "real 
worid"  applications  of  computer-based  instruction; 
including:  computer-assisted  interactive  video, 
multimedia,  hypermedia  and  learning  environments. 
Additionally,  these  types  of  projects  provide  tools  for 
ongoing  research  about  effective  instructional  design. 
As  the  Ph.D.  program  continues  to  expand,  the 
department  needs  content  areas  to  supply  design 
and  development  projects  as  a  basis  for  doctoral 
student  training  and  ongoing  research. 

Collaboration 

The  solution  for  each  program  Is  to  collaborate  on  the 
design,  development  evaluation  and  implementation 
of  interactive  multimedia  technology  which  provides 
introduction  and  practice  on  the  more  basic  concepts 
"and  skills  of  analysis,  freeing  classroom  time  for 
more  advanced  instruction.  This  partnership  began 
with  Mathias'  (1990)  doctoral  dissertation  which 
examined  interactive  videodisc  applied  to  swimming 
stroke  analysis.  This  interest  led  to  the  development 
of  the  Swimming  Stroke  Analysis  videodisc  program 
(Mathias  and  Smith,  1990).  Two  projects  followed 
shortiy  after,  Tennis  Skill  Analysis  (Chung,  Munson, 
Smith,  &  Phillips,  1992)  and  the  PETAI  Training 


Tutorial  and  Database  of  Pedagogical  Knowledge  in 
Physical  Education  (Horton,  Gregg,  Phillips,  &  Smith, 
1992).  In  light  of  the  unique  aspects  of  this 
collaboration,  this  paper  will  describe  the  design, 
development,  evaluation  and  implementation  of  the 
PETAI-IV  as  a  model  of  interactive  multimedia 
program  production. 

The  Physical  Education  Teacher  Assessment 
Program  (PETAI) 

The  PETAI  was  designed  to  measure  alterable 
teacher  and  student  behaviors  exhibited  in  the 
physical  education  classroom  (Phillips  &  Cariisle, 
1983).  The  model  from  which  the  PETAI  was 
developed  consisted  of  three  teacher  and  three 
student  behavior  categories  and  their  corresponding 
subparts.  The  scores  for  two  teacher  and  t^^'0 
student  behavior  categories  are  obtained  by 
recording  continuous  time  intervals  for  each  behavior 
change.  This  is  accomplished  by  coding  a  videotape 
of  ttie  class  session  using  the  computerized  version 
of  the  PETAI  (Phillips,  Cariisle,  Steffen,  &  Stroot, 
1986).  As  an  educational  tool  for  physical  educator 
training  at  the  University  of  Northern  Colorado,  the 
PETAI  is  initially  introduced  to  physical  education 
majors  who  enroll  in  a  required  junior  level  10-credit- 
hour  Methods  of  Teaching  and  Evaluation  Block. 
The  students  ultimately  use  the  PETAI  to  measure 
their  own  teaching  progress  during  their  field 
experiences.  In-service  physical  educators  have 
also  gained  exposure  to  the  PETAI  through 
wori^shops,  conferences,  satellite  colleges  and 
physical  education  pedagogy  graduate  courses 
offered  at  the  University  of  Northern  Colorado. 

Following  from  the  successful  use  of  the  previous  IV 
programs  in  the  physical  education  program  and 
based  upon  the  need  for  more  efficient  means  of 
delivering  insti'uction,  professors  in  each  of  the 
collaborating  programs  initiated  a  plan  to  design  and 
develop  an  interactive  video  ti*aining  tutorial  for  using 
the  PETAI. 

Design  Parameters  for  the  PETAI-iV 

Given  tlie  broad  range  of  potential  users  of  the 
PETAl-IV  a  two  phase  program  of  design  and 
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development  was  formulated.  In  Phase  1,  as  part  of  a 
doctoral  dissertation  study,  the  PETAI-IV  would  be 
field  tested  in  an  undergraduate  physical  education 
methods  course.  Based  on  the  results  of  this  field 
test,  Phase  2  would  expand  the  PETAl-IV  into  a  more 
robust  learning  instrument  that  could  be  used  in  the 
classroom  ur  as  a  reference  resource.  The  initial 
instructional  goal,  for  Phase  1 ,  v/as  to  train  the 
student  to  use  the  computerized  PETAI  for 
identification  and  coding  :f  teacher  and  student 
behavior  variables  as  defined  by  the  Physical 
Education  Teacher  Assessment  Instrument  (Phillips 
&  Cariisle,  1983).  However,  since  the  evaluation  of 
Phase  1  was  conducted  as  part  of  a  doctoral 
dissertation,  in  order  to  satisfy  the  requirements  of 
the  dissertation  research,  two  additional  goals  were 
adde(^;  understanding  PETAI  content  knowledge  and 
understanding  pedagogical  knowledge  in  physical 
education. 

Chen  (1990)  listed  three  obstacles  that  discourage 
the  implementation  and  use  of  IV  in  education:  1) 
limited  amount  and  lack  of  quality  of  IV  programming, 
2)  prohibitively  high  price  of  IV  systems  and 
interactive  program  production,  and  3)  teacher  and 
staff  resistance  to  implementation  and  integration  of 
IV  instruction  into  the  curriculum.  To  make  every 
conceivable  effort  necessary  to  increase  the 
likelihood  of  the  successful  development  and 
implementation  of  the  PETAI-IV,  a  design  team  was 
selected,  each  with  particular  skills  that  addressed 
each  of  these  three  obstacles. 

The  PETAI-IV  team  was  composed  of  a  content 
expert,  authoring  languages/computer  expert,  video 
technician,  and  a  facilitator.  The  content  expert  and 
the  facilitator  were  from  the  School  of  Kinesiology  & 
Physical  Education  and  the  computer  and  video 
experts  were  housed  in  the  Educational  Technology 
department.  Each  team  member  contributed  to  the 
instructional  design,  although  primary  duties  were 
shared  by  the  content  and  video  experts.  The  team 
members  from  the  Physical  Education  department 
had  no  formal  training  in  instructional  systems  design 
(ISD)  but  were  experts  in  the  area  of  pedagogical 
knowledge  in  Physical  Education,  whereas  the  team 


members  fi'om  the  Educational  Technology 
department  were  experts  in  instructional  design. 

Smith,  in  his  1987  review  of  the  use  and 
implementation  of  interactive  video,  stated  that  there 
had  been  litUe  discussion  of  interactive  video  design 
models  in  the  literature  perhaps  due  to  the  implicit 
assumption  that  "any  good  instructional  design  model 
can  be  adapted  to  the  medium"  (p.  7).  Furthermore, 
he  indicated  that  in  the  absence  of  a  specific  design 
model  IV  designers  rely  on  experience  in  media 
production,  CBI.  and  instructional  design.  Tripp  and 
Bichelmeyer  (1990)  proposed  an  approach  to 
instructional  design  for  computer-based  software 
using  the  principles  of  rapid  prototyping.  They  listed 
several  situations  for  which  rapid  prototyping  might 
be  appropriate  including  design  problems  for  which 
conventional  methods  have  proven  unsatisfactory 
and  new  design  situations  in  which  an  abundance  of 
experience  is  lacking. 

The  first  design  and  development  phase  of  the 
PETAi-IV  was  initiated  using  traditional  instructional 
design  methodology.  Several  videodisc  design  and 
project  management  models  were  used  as  the  basis 
for  'design,  development  and  project  management 
decisions  (Hooper  &  Hannafin,  1988;  Kelly,  Walkley, 
&  Tarrant,  1988;  Tarrant,  Kelly,  &  Walkley,  1988; 
Sti'ohmer,  1 988).  Growing  pressure  due  to  the 
consti'aints  of  time  and  budget  led  the  design  team  to 
shift  to  a  more  pragmatic  and  efficient  macro-design 
methodology,  employing  many  of  the  principles  of 
rapid  prototyping.  For  Phase  2,  the  design  team  will 
use  rapid  prototyping  methodology  as  a  model  for 
design  and  development  instead  of  more  traditional 
instructional  design  methods.  Hooper  and  Hannafin's 
(1988)  ROPES  model  will  guide  micro-design 
decisions. 

Design  assumptions  and  solutions  were  based  upon 
current  knowledge  in  pedagogy  in  physical  education 
and  cognitive  learning  theory.  Learning  from  media, 
according  tc  Salomon  (1983),  is  a  function  of  the 
degree  of  mindful  engagement  by  the  learner 
throughout  insti'uction.  Based  upon  several  reviews 
of  IV  design  (Chen,  1990;  Cronin  &  Cronin,  1992; 
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Smith,  1987),  the  PETAI-IV  design  team  made 
specific  design  decisions  regarding  attributes  of 
interactive  video  thought  to  promote  active 
involvement  and  deep  processing  on  the  part  of  the 
learner,  including:  1)  degree  of  interactivity,  2)  learner 
control,  and  3)  feedback. 

Development 

Several  factors  influenced  the  design  and 
development  process: 

•  time  limitations, 

•  budget  constraints, 

•  restrictions  to  meet  needs  of  the  dissertation 
research,  Phase  1  goals  and  Phase  2  goals, 

•  Windovys-based  platform  requirements. 

The  ultimate  goal  of  the  PETAI-IV  was  to  use  the 
characteristics  of  interactive  video  to  produce  a  high 
quality  teacher  preparation  educational  tool  which 
could  be  implemented  at  various  stages  in  a 
teacher*s  career.  However,  the  first  step  was  to 
design  a  training  tutorial  that  met  the  needs  of  the 
field  test  dissertation  research.  It  wss  determined 
that  a  highly  flexible  authoring  environment  would  be 
necessary  to  accomplish  both  directives. 
Furthermore,  at  the  outset,  given  the  timeline  for 
development  and  evaluation  and  the  relative 
inexperience  of  the  primary  development  team 
members,  a  traditional  instructional  systems  design 
approach  was  abandoned  as  being  too  time 
consuming  and  possibly  inefficient  Again,  a  more 
flexible  approach  was  adopted  using  the  principles  of 
rapid  prototyping. 

ToolBook  by  the  Asymmetrix  Corporation  was 
chosen  as  the  authoring  environment  since  it  met  the 
requirements  for  plasticity  and  modularity  needed  for 
rapid  prototyping  (Tripp  &  Bichelmeyer,  1990)  and  it 
was  Windows-based  software.  Software 
development  using  an  authoring  environment  such 
as  ToolBook  or  HyperCard  can  be  time  consuming. 
However,  the  design  team  estimated  that  use  of  the 
rapid  prototype  design  model  would  more  than  offset 
the  additional  software  development  time.  The 
choice  of  a  flexible  authoring  environment  was 
validated  when  a  pilot  test  of  the  PET AI-IV  led  to 


restructuring  the  sequencing  of  program  modules. 
Such  a  last  minute  revision  could  not  have  been 
accomplished  using  most  types  of  authoring 
software. 

Hoekema  (1983)  stated,  based  upon  familiarity  with 
television,  viewers  expected  broadcast  quality  video 
production  value.  This  level  of  video  production  was 
not  possible  for  this  project  However,  an  analysis  of 
the  decision  making  process  for  coding  the  PETAI 
revealed  that  in  most  cases  the  salient  information 
that  indicates  a  switch  in  behaviors  was  verbally 
indicated.  Thus,  even  using  a  video  acquisition 
format  of  mediocre  technical  specifications  (sVHS), 
the  designers  were  able  to  produce  a  videodisc  of 
acceptable  quality  by  paying  particular  attention  to 
shot  selection  and  use  of  color  to  focus  attention 
within  the  fi-ame,  and  by  recording  broadcast  quality 
audio. 

The  final  form  of  the  PETAI-IV  was  built  around  an 
instructional  unit  including  definition,  video  examples 
and  related  pedagogical  knowledge  for  each  of  the 
teacher  and  student  variables  as  defined  by  the 
PETAI.  A  brief  review  with  practice  was  placed  at  the 
end  of  each  major  ^^ection  of  teacher  and  student 
variables:  teacher  instruction  time,  teacher 
management  time,  student  participation  time,  and 
student  management  time.  Following  each  of  the  two 
major  components,  teacher  behaviors  and  student 
behaviors,  were  a  non-assisted  coding  practice 
exercise  and  a  guided  PETAI  coding  practice.  Each 
unit  allowed  for  learner  control  of  pace  and  review. 
Elaborative  feedback  was  provided  to  t!ie  learner  in 
response  to  learner  input  for  all  practice  items  within 
the  section  review  and  within  the  non-assisted  coding 
exercise.  The  non-assisted  coding  practice 
consisted  of  several  examples  of  teacher  or  student 
behavior  requiring  learner  response  after  each 
behavior.  After  incorrect  responses  the  learner  had 
the  option  to  try  another  input  to  repeat  the  stimulus 
video,  or  to  review  the  relevant  material.  Completion 
of  each  coding  practice  exercise  required  correct 
identification  of  the  teacher  or  student  behavior  in 
every  video  example. 
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Evaluation 

Ongoing  formative  evaluation  is  implicit  in  the  rapid 
prototyping  model  of  design.  This  type  of  evaluation 
was  used  throughout  the  development  of  Phase  1  of 
the  PETAI-IV.  As  was  stated  previously,  results  from 
such  an  evaluation  led  to  a  major  restructuring  of  the 
program  sequence  and  indicated  the  necessity  of 
adding  another  section  of  practice  coding  at  the  end 
of  the  training  tutorial. 

Norton  (1992)  has  reported  very  encouraging  results 
using  the  PETAI-IV  in  a  teacher  preparation  study. 
The  method  of  evaluation  was  a  media  comparison 
study,  a  type  of  study  that  has  been  criticized  in  the 
literature  (Clark,  1983).  Slee  (1989)  asked  "do  we 
really  need  to  compare  in  order  to  justify"  (p.  3).  As 
part  of  a  formative  evaluation,  a  media  comparison 
field  test  provides  important  data  for  decision-makers 
considering  implemention  of  a  new  instructional 
system  within  the  curriculum. 

Students  who  have  used  the  PETAI-IV  have 
perceiv  ,d  the  interactive  videodisc  instruction  as 
effective  and  useful.  Mnny  students  felt  that  they 
were  more  active  learners  while  using  the  IV  system. 
Furthermore,  the  students  felt  the  PETAI-IV 
adequately  prepared  them  to  properly  identify  and 
code  teacher  and  student  behaviors  using  the  PETAL 

Preliminary  analysis  investigating  the  use  of  the 
PETAI-IV  was  recentiy  initiated.  Results  indicated  a 
confirmation  of  the  Norton  study  using  another 
method  of  calculating  achievement  based  on  a 
cumulative  accuracy  score.  Using  a  modification  of 
audit  trail  recording  (Misanchuk  &  Schwier,  1992), 
the  IV  users  were  categorized  by  path  selection  and 
proportion  of  errors  in  practice  exercises.  People  who 
performed  with  fewer  errors  significantiy  differed  from 
those  with  high  error  performance  in  a  number  of 
areas  including  accuracy  and  attitude.  Analysis  is 
proceeding  to  determine  the  reasons  for  the 
differential  error  rate. 

Implementation 
The  evaluation  results  indicated  that  the  PETAI-IV 
tutorial  was  as  effective  as  traditional  classroom 


instruction  yet  required  much  less  instructional  time. 
Based  upon  these  results,  the  PETAI-IV  program  is 
being  integrated  into  the  preservice  teacher 
education  curriculum  of  the  School  of  Kinesiology  & 
Physical  Education.  Resti^ucturing  the  curriculum, 
even  at  such  a  small  scale,  will  not  be  accomplished 
without  difficulty.  Technical  support,  student  reaction 
and  cooperation,  and  faculty  support  have  already 
become  key  implementation  issues.  Even  with  the 
highest  quality  design  and  development  of  IV 
programming,  success  of  this  or  any  instructional 
interactive  video  program  depends  on  proper 
implementation  (Russell,  Sorge,  &  Campbell,  1991). 
As  the  design  of  the  PETAMV  moves  into  Phase  2, 
the  facilitator  assunries  a  critically  important  role 
within  the  design  team. 

Conclusion 

Phase  1  of  the  development  of  the  PETAI  Training 
Tutorial  and  Database  of  Pedagogical  Knowledge  in 
Physical  Education  (PETAI-IV)  took  only  five  months 
to  complete  with  actual  cash  outiay  totalling  less  than 
$1,000.00.  The  development  of  the  PETAI-IV  system 
took  a  commitment  from  a  team  of  individuals. 
Approximately  1,000  man-hours  of  time  was  devoted 
by  the  team  to  the  development  of  the  PETAI-IV 
system.  The  successful  collaboration  of  the  design 
team  has  created  new  opportunities  for  both 
programs.  The  physical  education  program  has  a 
validated  instructional  system  that  is  being  integrated 
into  their  curriculum.  The  educational  technology 
program  has  a  continuing  source  of  design  and 
development  projects  for  doctoral  student  training. 
Both  programs  are  benefitting  from  the  results  of  the 
ongoing  research  generated  by  these  projects. 
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Using  a  ToolBook®  Database  Front  End  to  Manage  Multimedia  Resources 

C.  Michael  Hassett,  Fort  Hays  State  University 
Mary  R.  Hassett,  Fort  Hays  State  University 


Abstract 

Managing  resources  for  multimedia  is  becoming  a  nontrivial  task.  The  presentation  describes 
and  demonstrates  a  database  approach  for  managing  multimedia  resources  using  an 
Asymetrix  Multimedia  ToolBook®  database  front  end.  The  resources  demonstrated  include 
different  content  formats  such  as  text,  graphics,  animation,  audio,  still  images  and  motion 
video.  Storage  media  includes  optical  devices  (e.g.,  CD-ROM,  laser  videodisc),  a  PC-VCR, 
and  an  audio  library.  A  new  term,  resourcebase,  is  proposed  for  multimedia  and  Microsoft's 
RIFF  format  is  advocated  for  file  types. 

Background 

Resources  available  for  multimedia  are  popping  up  so  fast  that  managing  them  is  a 
nontrivial  task.  Traditionally,  such  resources  were  fairly  easy  to  manage.  For  example,  a 
computer-based  video  instruction  program  had  one  set  of  resources.  The  program  "owned" 
that  set  of  resources  and  the  format  and  access  path  were  a  part  of  the  program  (see  Hassett, 
Hassett  &  Unger,  1991).  With  current  multimedia,  a  given  resource  may  be  shared  by 
numerous  programs;  there  is  a  growing  independence  of  resources  and  programs.  One  may 
wish  to  use  the  same  audio  resource  in  two  different  multimedia  programs  and  be  unable  to 
locate  the  original  resource.  If  one  has  audio  libraries,  graphics  libraries,  scanned  images, 
etc.,  finding  a  particular  resource  in  real  time  is  often  difficult. 

The  impetus  for  this  paper  came  in  1991  with  the  proliferation  of  multimedia  resources, 
such  as  those  on  CD-ROM,  and  the  overv/helming  task  of  keeping  track  of  thern.  Resources 
such  as  bitmap  graphics  and  audio  waveforms  were  created,  modified  and  merged  as 
applications  were  developed,  but  without  history  or  documentation.  Later,  when  these 
resources  needed  updating  or  modification,  things  got  sticky.  For  example,  what  might  start 
off  as  a  prototype  could  evolve  into  a  final  product.  Without  a  method  to  link  various 
resources  to  their  origins,  or  documentation  on  how  the  resources  were  created,  issues  such 
as  copyright  ownership  suddenly  became  important.  Also,  the  sheer  number  of  resources 
became  unmanageable.  Miller  noticed  a  difference  between  end-user  functions  for  a 
videodisc  player  versus  CD-ROM  (Miller,  1992).  One  can  easily  search  videodiscs  for 
chapters  or  frames.  For  CD-ROM,  however,  such  end-user  functions  are  not  readily  available. 

The  database  approach  addresses  problems  with  data  management.  Since  a  similar 
phenomenon  has  taken  place  with  data  as  corripared  to  multimedia  resources,  a  review  of  the 
database  approach  became  the  starting  point. 
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Database  Approach 

Databases  are  integrated  collections  of  automated  data  file?  related  to  one  another  to 
support  a  common  purpose  (Stevens,  1991).  The  database  approach  views  data  as  being 
shared  by  a  number  of  applications  rather  than  being  exclusively  owned  by  one  application. 
The  conceptual  (user)  view  of  the  data  is  independent  of  the  method  used  to  physically  store 
the  data.  The  three  most  popular  conceptual  models  are  the  relational,  hierarchical  and 
network.  And  more  recently  object  oriented  models  have  been  proposed.  The  relational 
model  is  used  as  a  basis  for  a  number  of  database  management  systems  such  as  Borland's 
dBASE  IV. 

It  became  evident  that  the  approach  needed  to  managing  multimedia  resources  is  very 
different  from  the  approach  used  so  successfully  for  managing  data.  The  paper  describes  a 
proposed  approach  for  managing  multimedia  resources  that  borrows  from  techniques  used  in 
the  database  approach:  resourcebase.  The  Asymetrix  Multimedia  ToolBook  is  a  software 
construction  set  for  building  multimedia  applications  in  the  Microsoft  Windows  3.0 
environment  (Asymetrix,  1991b),  Multimedia  ToolBook®  was  used  to  build  a  database  front 
end  and  rapid  prototyping  was  used  to  develop  the  approach  (Hassett,  1992).  The  multimedia 
resources  include  '"ferent  content  formats  such  as  text,  graphics,  animation,  audio,  still 
images  and  motion  vrdeo.  Storage  media  include  CD-ROM  graphics,  laser  disc  video  and  an 
audio  library. 

Traditional  Databases 

Traditional  databases  are  comprised  of  data  represented  by  numeric,  aphhanumeric  and 
symbol  (the  smallest  unit  of  storage).  A  traditional  database  storage  hierarchy  consists  of 
these  elements:  SYMBOL  ^  FIELD  ^-  RECORD  ^  FILE  ^  DATABASE  (see  Table  1 ).  For 
example,  a  fictitious  record  company,  FASTRAK  Records,  may  have  a  database  called  CD,  in 
which  are  listed  all  compact  disk  titles,  type  of  music  (e.g.,  contemporary,  pop,  country)  the 
artists,  the  titles  of  the  recordings  ("cuts")  on  each  disk,  the  recording  label  (record  company 
name),  etc.  Each  file  contains  records  (rov/s)  and  fields  (columns).  A  typical  query  of  the 
database  might  be  to  list  all  the  recordings  by  a  particular  artist  and  the  names  of  the  title  CDs 
that  carry  these  recordings.  A  database  front  end  created  in  ToolBook  would  provide 
FASTRAKW\\h  friendly  end-user  functions  (Asymetrix,  1991a). 

Database  Front  End 

A  ToolBook  database  front  end  provides  instant  access  to  functionality,  accessing 
data  with  a  Dynamic  Link  Library  (DLL)  or  Dynamic  Data  Exchange  (DDE)  (Asymetrix, 
1991a).  ToolBook  ships  with  a  sample  database  front  end  that  provides  access  to  dBASE 
or  dBASE  Ml  files.  (One  does  not  need  to  own  dBASE  to  use  the  DLL.)  The  database 
front  end  provides  the  following: 

1 .  maintenance  of  data  in  its  original  form; 

2.  an  easy-to-use  functions  with  the  database,  including: 

a.  ways  to  view  all  data 

b.  ways  to  reorganize  the  data  to  be  viewed 
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c.  ways  to  add  new  data  or  modify  existing  data;  and 
3.  hiding  the  complexity  of  the  database  manipulation  language. 


Resourcebase  Approach  for  Multimedia 

A  new  term,  resourcebase,  is  proposed  as  a  new  approach  for  multimedia  management. 
Resourcebases  are  integrated  collections  of  automated  directories  containing  tagged 
resource  files  and  related  to  one  another  to  support  multimedia  (Hassett,  C.  M.,  1992). 
The  smallest  unit  of  storage  in  a  resourcebase,  as  in  a  database,  is  a  symbol.  Observing 
multimedia  resources,  it  is  proposed  that  the  resourcebase  storage  hierarchy  consists  of  these 
elements:  SYMBOL  ^  RIFF  CHUNK  ^  FILE     DIRECTORY  ^  RESOURCEBASE  (see 
Table  1).  For  example,  an  author  may  have  a  multimedia  resource  called  MM,  in  which  are 
listed  all  audio  objects  (e.g.,  sound  effects,  music),  graphics  objects  (e.g.,  clip  art,  figures, 
scanned  images),  animation  (e.g.,  multimedia  movies),  commonly  used  text  objects,  and  video 
objects  (e.g.,  digitized  motion). 


Table  1. 

Resourcebase  vs.  Database 


Resource- 

Database 

base 

•DIRECTORY 

•FILE 

•FILE" 

•RECORD 

•RIFF  CHUNK 

•FIELD 

•SYMBOL 

•SYMBOL 

Resourcebase:  C.  Michael  Hassett,  1992 


Prototype:  Resourcebase 

The  work  station  for  the  prototype  consisted  of  the  following:  IBM  PS/2  Model  95 
microcomputer  with  IBM  M-Audio  and  M-Motion  boards,  a  NEC  MultiSync  5FG  display,  a 
Pioneer  CD-ROM  Changer  DRM-600,  an  HP  LaserJet  lllsi  printer. 

Primary 

Primary  multimedia  resources  include  those  that  are  device  dependent  and  those  device 
independent  (Figure  1 ).  Device  dependent  resources  include:  CD  audio,  laser  videodisc  and 
audiocassette  tapes.  The  prototype  uses  all  the  device  dependent  and  independent 
resources. 
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DEVICE  DEPENDENT 


DEFACE 
INDEPENDENT 


Figure  1.  Primary  multimedia 
resources: 

x'Developed  by  C.  Michael  Hassett,  1992 


Secondary 

Secondary  multimedia  resources  include,  digitized  images  (e.g.,  photographs),  and 
digitized  audio  (e.g.,  recordings).  Secondary  resources  often  require  special  handling.  One 
excellent  advent  is  the  audio  library,  such  as  the  Prosonus  MusicBytes^"  (Prosonus,  1 991 ). 
MusicBytes  provides  excellent  music,  sound  effects  and  MIDI  resources. 
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Resource  Interchange  File  Format  (RIFF) 

In  a  traditional  database,  similar  data  are  grouped  together  as  records  in  one  file. 
Multimedia  resources,  however,  presently  require  one  file  per  resource.  The  resource 
must  be  referenced  using  the  file  name;  in  the  Microsoft  DOS/Windows  environment,  this 
means  eight  characters  for  the  name  and  three  characters  for  the  extension.  With  this 
limitation,  one  must  be  quite  creative  to  properly  manage  even  a  small  number  of 
resources.  What  was  needed  was  a  way  of  attaching  the  reference  data  to  the  file.  The 
Microsoft  RIFF  file  format,  along  with  a  complete  set  of  I/O  (Input/Output)  services, 
provided  the  solution.  Resource  Interchange  File  Format  (RIFF)  is  a  platform-independent 
multimedia  specification  allowing  audio,  graphics  and  other  multimedia  to  be  stored  in  a 
common  format  (Miller,  Sayers,  Reeve  &  Kasten,  1991).  The  first  attempts  to  manage 
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resources  using  file  names  and  indexing  was  abandoned  as  soon  as  RIFF  became 
available  (Figures  2  &  3). 

The  RIFF  file  format  is  a  tagged  file  structure  that  allows  chunks  of  data  (called  "RIFF" 
chunks;  see  Table  2  for  sample  registered  chunk  IDs)  to  be  stored  with  multimedia 
resource  data  (sound,  graphics,  multimedia  movies,  palettes)  in  one  file.  RIFF  forms  are 
registered  with  Microsoft  Corporation.  A  list  of  sample  registered  forms  is  found  on 
Table  3. 

RIFF  has  a  counterpart,  RIFX  that  can  be  used  to  define  RIFF  file  formats  that  use  the 
Motorola  format  rather  than  the  Intel  format.  Like  RIFF  the  RIFX  forms  must  be 
registered.  The  Windows  3.1  Software  Development  Kit  (SDK)  provides  a  number  of 
functions  for  managing  RIFF  I/O  services.  For  example  the  functions  mmioAscend  and 
mmioDescend  ascend  and  descend  the  chunks  of  a  RIFF  tagged  file  structure. 


RIFF  FILE 

|USTCHUNK  ^1 

Figure  2.  Resource  Interchanqe 
File  Format  (RIFF).  ^ 

Microsoft  Corporation.  (1992K  Murtl media  phxirammer's  qutde. 

INFO  1 

[chunk  j-*^ 

m 
m 
m 

Figure  3,  RIFF  INFO  chunk. 

Microsoft  CoroorMlon.  (1992).  Muttim«dla  oroarammer's  guide. 
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Table  2. 

Sampie  Microsoft  Four 
Character  INFO  Chunk  IDs 


Char 

INFO  Chunk  ID 

lARL 

Archival  Location 

lART 

Artist 

ICMT 

Comments 

ICOP 

Copyrigtit 

ICRD 

Creation  Date 

IDs  Registered  by  Microsoft.  1992. 


Table  3. 

Registered  RIFF  Forms 


Form 

Type 

Description 

PAL 

Palette  File  Format 

RDIBRIFF 

Device  Independent  Bitmap  Format 

RMIDRtFF 

MIDI  Fomiat 

RMMP 

RIFF  Multimedia  Movie  Format 

WAVE 

Waveform  Audio  Format 

Registered  by  Microsoft,  1992. 


Convert  and  FileWalker 

The  Microsoft  Multimedia  Development  Kit  (MDK)  provides  a  set  of  file  editing  tools. 
The  Convert  and  FileWalker  tools  were  used  to  prepare  files  for  the  resourcebase.  The 
Convert  tool  was  used  to  convert  non  RIFF  files  to  RIFF  format.  The  FileWalker  is  an 
editor  that  performs  the  following  tasks:  (a)  view  the  outline  and  data  structure  of  a  file, 
(b)  create  new  structures,  and  (c)  edit  file  data.  The  FileWalker  was  used  to  create  the 
RIFF  chunks  and  input  the  data  that  is  depicted  on  Figure  4.  Future  implementations  of 
the  book  will  incorporate  these  functions  and  the  FileWalker  will  not  be  needed. 

Implementation:  ToolBook  Front  End  Prototype 

Asymetrix  Multimedia  ToolBook  proved  to  be  an  excellent  tool  for  developing  a 
resourcebase  (multimedia  database)  front  end.  The  Widgets  Book  provided  the  controls 
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for  managing  most  multir  ledia  resources.  When  the  controls  (buttons,  fields,  etc.)  were 
copied  to  the  new  book,  the  scripts  were  included^  Only  minor  debugging  was  required 
to  make  the  controls  functional  in  the  new  book.  A  major  advantage  of  ToolBook  is  that 
features  can  be  added  without  making  major  revisions  in  existing  features.  For  example, 
controls  for  viewing  reference  information  in  an  audio  file  such  as  Author,  Copyright, 
etc.,  was  added  to  a  book  page.  Later,  controls  were  added  to  audition  the  audio.  This 
type  of  activity  was  essential  for  the  rapid  prototyping  development. 


Eile    Edit   Eage  Help 


PArchivalLocation:  File  cabinet  (home  offc)  under 
PUBLISHING:  PHOTOS  of  authors 
Artist:  Jack  Jackson 
Commissioned:  Mary  R«  Hassett 
Comments:  For  publications^  news  releases^  scanning 
syllabi  for  distance  learners 
Copyright:  None 
CreationDate:  1991-05-01 
Cropped:  Proponional 

smenslon  2  in.     3  in.  h 
DotsPerinch:  600 

Engineer  Hassett.  C.  Michael;  Hassett^  Mary  R. 
Genre:  portrait 

Keywords:  portrait;  Mary  Hassett 
Lightness:  Missing  da\a 
Medium:  bSack  &  white  glossy  photograph 
Name:  Mary  R.  Hassetl  bust 
PalcttcSettIng:  2 
Product:  Hays  Daily  News 


Figure  4,  MSVI  Resource  Manager  page. 

Created  w/ Asymetrix  MM  TookBook  by  C.  M-  Hassett  &  M.  R.  Hassett,  1992. 


Dynamic  Link  Libraries 

ToolBook  did  not  provide  I/O  (Input/Output)  services  for  the  Microsoft  RIFF  file 
format.  It  was  necessary  to  write  a  Dynamic  Link  Library  (DLL)  to  provide  ToolBook 
access  to  the  RIFF  files.  Writing  the  DLL  required  a  Microsoft  C  compiler  and  Microsoft 
Software  Development  Kit  (SDK).  The  Borland  C  compiler  can  also  be  used  to  write 
DLLs*  The  Microsoft  C+  +  version  7.0  and  Microsoft  Windows  3.1  were  used.  SDK 
was  used  to  write  the  final  DLL.  A  good  starting  place  when  writing  DLLs  for  ToolBook 
is  to  download  the  notes  and  examples  from  the  Asymetrix  Bulletin  Board  Service.  The 
gnosis.zip  example  file  contained  a  sample  ToolBook  book  that  used  a  DLL,  a  compiled 
version  of  the  DLL  and  a  Microsoft  C  version  6.0  source  code  version.  Only  minor 
changes  were  necessary  to  compile  the  DLL  using  the  new  Microsoft  7.0  C+  -f  compiler. 
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This  source  code  version  of  the  DLL  was  used  as  a  "shell"  and  the  required  RIFF  I/O 
functions  were  added  to  support  the  resourcebase  front  end  ToolBook,  little  work,  it  can 
be  done,  the  ability  to  write  DLLs  greatly  extends  the  opportunities  available  to  a 
Microsoft  Windows  user. 


Using  the  ToolBook  Front  End  Prototype 

The  ToolBook  Front  End  Prototype  is  organized  into  ToolBook  pages,  based  on  the 
type  of  resource  and  the  management  function.  For  example  the  page  in  Figure  4  allows 
the  user  to  select  a  directory  and  then  view  all  the  bitmap  resources  in  that  directory. 
Additional  pages  provide  resource  management  for  special  requirements.  For  example 
the  resourcebase  might  contain  person  photos  (bitmap),  person  recorded  names 
(waveform)  and  person  signed  signatures  (bitmap).  The  user  can  select  a  person  photo 
directory  to  view  ail  the  photos  and  related  signatures.  While  a  given  photo  is  displayed 
the  recorded  name  related  to  that  photo  and  signature  can  be  auditioned.  The  user  can 
initially  select  any  of  the  three  resources  and  the  ToolBook  Prototype  will  automatically 
match  the  other  two  related  resources. 


Conclusions 

The  RIFF  file  format  has  provided  a  partial  solution  to  managing  multimedia  resources 
(resourcebases).  What  is  lacking  is  registered  forms  for  video,  it  is  expected  these  will  be 
forthcoming  when  video  is  commonly  digitized  (e.g.,  with  Digital  Video  Interactive,  DVI).  The 
authors  recommended  that  the  concept  of  resourcebases  for  multimedia  be  tested  and 
evaluated. 
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A  PRIMER  ON  DVI:  DIGITAL  VIDEO  INTERACTIVE 


Dr.  D.K  Pigonl 
Western  Kentucky  University 

ABSTRACT 

This  paper  on  Digital  Video  Interactive  will  highlight  the  basic  technical  details  and  applications 
of  this  new  technology,  DVI  is  a  hardware/software  platform  designed  to  foster  the  rapid 
transrai^^sion  of  digital  video/audio  on  Intel  386  or  Macintosh  personal  computer  networks. 
Topics  covered  will  include  a  brief  overview,  developmental  histor>',  hardware  basics,  software 
implementations,  and  project  applications. 

OVERVIEW  OF  DVI 

Digital  Video  Interactive  or  DVI  is  a  technology,  not  a  system.  This  technology  allows  for  the 
storage/display  of  audio  and  video  files  for  enhancing  multimedia  platforms.  The  all  digital 
formats  from  a  DVI  system,  unlike  the  interactive  videouisks  systems,  allow  for  the  editing, 
transforming,  and  sending  of  audio/video  files  across  a  microcomputer  network. 

A  DVI  system  consists  of  four  components:  a  set  of  custom  VLSI  chips,  audio/video  file 
specifications,  formats  for  a  run-time  interface,  and  a  set  of  compression/decompression 
algorithms.  These  components  are  commercially  available  as  Action  Media  II  products  from 
Intel/IBM  and  the  EyeQ  board  from  New  Video  Corporation  for  the  Macintosh.  From  a  user^s 
perspective,  a  typical  DVI  system  is  comprised  of  a  personal  computer  or  workstation  with  Intel's 
Action  Media  II  playback  card,  system  software,  and  a  CD-ROM  drive.  With  the  addition  of  the 
capture  card  and  peripherals  (videotape  player,  audio  cassette  player  or  camera),  a  user  can 
create  multimedia  elements  (motion  video,  3D  graphics,  and  multitrack  audio)  for  display/play. 

DVI  multimedia  products  are  currently  being  developed  on  the  following  personal  computer 
platforms: 

Operating  Systems:  OS/2,  Windows  3.1,  DOS 
Computer  Bus:  AT,  Micro  Channel,  NUBus  (Macintosh) 
Processor:  Intel  386  SX,  Intel  386/DX,  Intel  486,  Motorola  6800 
Media  Format:  CD-ROM,  Hard  Disk,  Network 
Graphics:  VGA,  XGA 

Network  Support:  Token  Ring,  Ethernet,  Fiber  Optic 
Since  these  multimedia  elements  are  all  in  a  digital  format,  they  can  be  stored  on  any  random 
access  storage  device  and  even  transmitted  across  a  network.  The  DVI  platform,  moreover, 
allows  content  to  be  developed  once  and  employed  in  several  environments. 

DEVELOPMENTAL  HISTORY  OF  DVI 

Realizing  the  potential  for  an  all  digital  multimedia  system,  the  Radio  Corporation  of  America 
(RCA)  began  such  investigations  in  1983  at  its  central  research  laboratory,  the  David  Sarnoff 
Research  Center  in  Princeton,  N.J.  Even  though  a  team  of  fifty  people  headed  by  Arthur 
Kaiman  worked  on  the  project  development,  Larry  Ryan  is  usually  termed  the  "Father  of  DVI". 
In  1986  RCA  sold  DVI  rights  to  General  Electric.  At  that  time  the  DVI  technology  was  capable 
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of  playing  full  motion  digital  video  from  a  CD-ROM.  In  March  1987  DVI  was  shown  publicly 
at  the  2nd  Microsoft  CD-ROM  Conference  in  Seattle,  WA. 

In  November  1990  Intel  introduced  a  set  of  components  (the  1750  video  processor)  which  allowed 
manufacturers  to  add  digital  video  and  audio  to  personal  computers.  In  October  of  the  next  year 
Intel  and  IBM  introduced  powerful  cost  effective  add-in  boards  (based  on  an  upgraded  version 
of  the  same  video  processor)  called  Action  Media  II  boards  to  foster  multimedia  software 
development  across  the  OS/2  and  Windows  environments.  Currently  the  ActionMedia  II  boards 
cost  about  $1900  and  the  Macintosh  DVI  boards  run  about  $2500. 

HARDWARE  BASICS  OF  DVI 

From  a  hardware  perspective,  a  DVI  video  system  is  composed  of  the  following  components: 
82750PB  Pixel  Processor 
82750DB  Display  Processor 
Video  RAM 

Host  Microprocessor  Interface  Logic 
VRAM  timing  generator 
A  video  digitizer  and  video  buffer/mixer  are  optional  components. 

The  pixel  and  display  processors  are  based  on  the  i750  chip  set.  The  82750  RB  pixel  processor 
runs  its  own  microcode  to  perform  operations  on  digital  images  at  12.5  MIPS  (Millions  of 
instructions  per  second).  The  processor  can  compress/decompress  image  data  from  memory  to 
generate  fast  graphics  and  special  effects  at  a  rate  of  2.5  million  16  bit  pixels  per  second. 

The  82750  DB  display  processor  takes  data  from  a  bitmap  in  VRAM  and  converts  it  to  a  digital 
output  signal  which  feeds  a  triple  D/A  converter  to  drive  a  color  RGB  monitor.  The  possible 
outputs  are  compatible  with  NTSC  or  PAL  standards.  The  bpp  (bits  per  pixel)  ranges  across  8, 
16,  and  32  with  screen  resolutions  up  to  1024  by  512.  The  video  resolution  is  256  by  240.  The 
display  processor  allows  the  screen  resolution,  active  pixel  area,  video  and  synch  formats,  screen 
border,  interrupt  generation,  and  color  map  entities  to  be  programmed  [Luther89]. 

The  hardware  handles  both  still  video  images  and  video  sequences  (movies).  A  still  image  file 
(512  by  480)  ranges  from  15-35K  in  the  JPEG  standard  (Joint  Photographic  Experts  Group). 
A  PIC  file  format  for  video  images  ranges  from  80-1 15K  or  150-250K.  The  PHOTO  CD  Image 
format  used  by  Kodak  for  its  PHOTO  CD  products  is  also  supported.  The  video  sequences  are 
much  larger.  It  takes  18  million  bytes  of  data  moved  per  second  to  produce  30  frames  of  video 
per  second.  The  hardware  compresses  each  frame  to  150,000  bytes  on  CD-ROM. 

The  algorithms  for  the  compression  of  video  frames  throw  out  the  redundant  pixel  information 
keeping  only  new  information  relevant  to  the  previous  frame.  The  algorithm  stores  only  one  out 
of  every  few  pixels  during  compression*  During  decompression,  an  interpolation  scheme  is  used 
to  recreate  the  missing  pixels.  These  schemes  along  with  current  improvements  enable  a  motion 
video  frame  to  be  compressed  to  a  ratio  of  160  to  1.  A  DVI  board  is  required  for  playback  of 
any  video/audio. 
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DVI  algorithms  support  two  modes  of  motion  video  today,  RTV  and  PLV.  RTV  (Real  Time 
Video)  is  produced  in  reaj  time  and  is  near  VCR  quality.  PLV  or  Production  Level  Video 
equals  VCR  quality  but  it  must  be  produced  offsite  by  a  licensed  vendor. 

In  summary,  the  DVI  hardware  supports  both  siill  and  motion  video  standards  and  uses 
compression/decompression  algorithms  to  effect  digital  video. 

SOFTWARE  DEVELOPMENT  FOR  DVI 

In  any  technology,  the  development  software  must  work  in  harmony  with  the  hardware.  The 
DVI  development  platform  consists  of  a  runtime  software  library,  runtime  include  files,  font  files, 
microcode  files,  driver  files,  and  "authoring  tools".  The  authoring  tools  are  termed  the  API  or 
video  Application  Programmer's  Interface.  These  tools  consist  of  high  level  C  function  calls  to 
the  runtime  software  along  with  lower  level  C  language  interfaces  and  a  VGRCMOD  interpreter. 
These  C  function  calls  control  the  videodisk  functions,  synthetic  video,  video  manipulation,  and 
high  speed  graphics. 

The  C  application  code  is  linked  with  four  system  submodules  for  graphics,  audio/video,  real 
time  execution,  and  standard  system  functions.  The  GR  submodule  handles  graphics  primitives, 
video  manipulation,  and  image  handling.  The  AVSS  A/V  unit  supports  the  playback  of 
audio/video  files  in  the  AVSS  format  and  the  audio/video  playback  from  CD-ROM,  hard  disk, 
or  RAM.  The  RTX  unit  provides  for  multitasking  inside  of  the  FVI  application  whereas  the 
STD  module  handles  system  functions  like  memory  management,  input/output,  and  error 
handling.  When  an  application  is  running,  the  video,  audio,  VRAM,  and  CD-ROM  drivers  are 
resident  in  memory  along  with  the  Microsoft  CD-ROM  extensions. 

Presently  the  DVI  authoring  tools  are  most  useable  by  trained  C  programmers;  however, 
authoring  systems  are  evolving  on  the  market  today.  Authology  from  Ceit  Systems  is  a  DVI 
application  tool  for  the  DOS  environment.  MediaScript  from  Network  Technologies  is  a  similar 
DOS  multimedia  authoring  tool.  Both  companies  plan  extensions  to  either  the  OS/2  and/or 
Windows  platforms.  MediaScript  utilizes  a  visual  programming  interface  for  high  level 
multimedia  applications.  In  the  Summer  of  92  IBM  announced  two  video  editing  packages, 
LinkWay  Live  and  Audio  Visual  Connection  for  the  Windows  3.1  environment.  The  applications 
interface  is  now  termed  the  AVK  or  Audio  Video  Kernel. 

With  an  all  digital  format  for  video  and  audio,  the  DVI  environment  is  a  natural  for  utilization 
in  a  networked  environment.  Network  support  software  called  Videocomm  for  Novell  Netware, 
DOS,  and  Windows  platforms  is  available  from  the  ProtoComm  corporation. 

As  the  software  tools  evolve  and  improve,  the  applications  development  will  definitely  increase. 
Project  Applications  with  DVI  With  the  advent  of  DVI  hardware  and  software  development, 
application  development  in  multimedia  will  spiral.  Applications  range  from  code  review  training 
at  Carneige-Mellon  University  to  weapons  maintenance  training  at  General  Electric  Advanced 
Technology  Lah.  The  following  chart  summarizes  some  initial  DVI  development  applications 
in  progress  [Ripley89]. 
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APPLICATION  AREA 


BETA  SITE 


Truck  Driver  Safety  Training 
Clothing  Store  Point  of  Sale 
Military  Intelligence 
Steel  Casting  Training 
Courseware  Development  Software 
Process  Control  Technology 
Imaging  Systems 

Submarine  Maintenance  Training 
Military/Government  Applications 
Image  Storage/ Analysis 
Movies  of  the  Future/Workstations 
Travel  Agency  Information 
Educational  Products/Exhibits 
Training  and  Merchandizing 
Paint  and  3-D  Animation 
Entertainment/Training 
Furniture/Interior  Design 
Virginia  Polytechnic  Institute 


Applied  Optical  Media 
Auteur,  Inc. 
Betac  Corporation 
Bethlehem  Corp,  &  Lehigh 
CBIS,  INC 
Dow  Chemical 
Eastman  Kodak 
General  Dynamics 
General  Electric 
LTS  Corporation 
M.  L  T. 
Comsell 

National  Geographic 
Spectrum  Interactive 
Time  Arts,  Inc. 
Video  Design  Group,  INC. 
Videodisk  Publishing,  Inc. 
Authoring,  Education 


This  list  of  project  applications  is  expanding  rapidly  as  more  and  more  business,  training,  and 
educational  multimedia  applications  are  developing. 

One  interesting  application  in  education,  codeveloped  by  the  Bank  Street  College  of  Education, 
is  entitled  Palenque.  This  interactive  multimedia  program  allows  children  aged  eight  to  fourteen 
to  explore  a  multimedia  database  about  the  ancient  Mayan  sites.  In  this  multimedia  application, 
360  degree  panoramic  views  are  accessible  under  full  user  control  at  interesting  points 
throughout  the  sites. 

SUMMARY 

Digital  Video  Interactive  or  DVI  is  a  technology  in  its  second  generation  of  development.  With 
its  combined  hardware  and  software,  the  DVI  environment  can  handle  audio  and  video  in  all 
digital  formats.  Once  in  this  all  digital  format,  compressed  files  can  reside  on  a  server  for 
playback  on  a  network  of  personal  computers.  With  the  emergence  of  recent  cost  effective 
commercial  products,  DVI  is  a  viable  multimedia  authoring  and  delivery  platform  for  training 
and  educational  endeavors. 
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The  Compatibility  Problem  in  Designing  and  Developing  Educational  Software  with  the 

New  Generation  of  Smart  Tools 

Emile  Attala  and  Tommy  DeMoville,  California  Polytechnic  State  University 

To  be  able  to  develop  educational  software  in  interactive  media  and  related  technologies,  the 
designer  needs  to  have  authoring  systems  that  can  accommodate  and  be  compatible  with  several 
standards  of  digitized  graphics  files  and  captured  video  files.  Authoring  systems  should  provide 
single  button  database  management  systems  that  can  maintain >  restore,  reclaim,  and  perform 
archive  backup  to  all  these  different  files. 

What  developers  face  today  is  working  with  graphics  file  standards  that  are  anything  but 
standard.  Part  of  the  problem  comes  from  the  attempts  to  maintain  compatibility  with  older 
formats.  This  would  allow,  for  example,  a  VGA  system  to  reproduce  CGA  and  EGA  images 
on  the  screen  and  allow  older  software  to  continue  to  run.  This  compromise  creates  conflicts 
between  the  formats  of  different  color  palettes  that  depend  upon  the  resolution  in  which  they 
were  created,  A  picture  done  at  one  resolution  with  a  given  color  palette  may  undergo  a  color 
rotation  when  brought  into  an  authoring  language  that  uses  a  different  resolution.  Cyan  may 
change  to  yellow,  red  to  blue,  brown  to  grey  and  the  resulting  output  to  the  screen  may  not  be 
recognizable  to  the  initial  designer. 

Resolution  conflicts  may  also  produce  pictures  that  are  stretched  or  compacted  along  one  axis 
or  another.  Just  going  from  an  original  640  x  480  resolution  picture  to  a  program  expecting  640 
X  350  input  would  result  in  a  multi-colored  garbage  screen. 

Two  paint  programs  rarely  produce  the  same  graphics  file  format,  A  PCX  file  is  not  equivalent 
to  a  GIF,  MAC  or  TIFF  file.  There  are  even  differences  between  PCX  file  formats  from  one 
company  to  another  or  even  from  the  same  company  with  a  different  release  date. 

Software  developers  have  tried  to  work  around  this  problem  by  creating  capture  programs  that 
can  grab  a  screen  and  output  a  file  in  the  format  needed  for  a  given  authoring  system.  They  still 
haven't  mastered  the  ambiguity  of  the  various  resolutions  and  color  palettes  produced  by  the 
variety  of  video  formats  available  now  on  the  market.  Fifth  generation  object-oriented  authoring 
systems  need  to  be  designed  to  eliminate  these  conflicts. 
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The  Application  of  Motivational  Principles  to  Visual  Screen  Design 

Charles  F.  Belinski  and  Jimmy  Williamson,  The  University  of  Georgia 

This  session  will  consist  of;  a  brief  description  of  the  ARCS  model,  a  description  of 
environmentally-centered  motivational  theory,  a  listing  of  ergonomic  considerations,  and  a  brief 
discussion  concerning  components  of  visual  design  and  instructions  on  integrating  these  factors 
when  designing  visuals.  The  emphasis  will  be  on  designing  appealing  and  effective  instructional 
materials  and  visuals. 

Using  the  ARCS  Model  as  a  starting  point,  a  taxonomy  of  motivational  characteristics  will  be 
offered  that  can  be  applied  to  visual  design.  Keller's  Model  as  well  as  other  motivational 
theories  will  be  used  to  derive  a  tool  for  applying  motivational  considerations  to  visual  design. 
An  emphasis  will  be  placed  on  screen  design.  However,  a  number  of  media,  both  still  and 
motion,  will  be  addressed. 

Basic  visual  ergonomics  will  be  addressed,  particularly  as  they  relate  to  screen  design. 
Considerations  such  as  eye  fatigue  and  visual  clarity  will  be  discussed  briefly.  The  discussion 
on  the  components  of  visual  design  will  briefly  define  concepts  such  as  shape,  color, 
perspective,  and  depth. 

The  bulk  of  the  presentation  will  consist  of  examples  of  visual  design  and  explanations  of  the 
motivational  factors  considered  when  creating  these  examples.  Demonstrations  of  the  various 
characteristics  will  be  done  in  three  formats;  still  visual,  motion  media  visual,  and  computer 
screen  visual.  A  special  emphasis  will  be  placed  on  the  application  of  motivational 
characteristics  for  designing  computer  screen  visuals  and  multimedia  applications. 


Computer  Adaptive  Testing:  Preparation  and  Analysis  of  the  Item  Pool 

Randal  J.  Carlson,  The  Pennsylvania  State  University 

The  purpose  of  this  study  is  to  report  on  the  calibration  and  analysis  of  an  item  pool  that  is  to  be  used 
in  a  larger  study  of  adaptive  testing.  Over  14,000  freshmen  completed  a  72  item  mathematics  placement 
test  at  a  large  eastern  university.  From  th^e  respondents,  a  random  sample  of  1000  was  drawn  from  the 
approximately  2000  respondents  that  had  answered  every  test  item.  Using  Item  Response  Theory,  1-, 
2-,  and  3-parameter  logistics  models  were  fit  using  the  PC  version  of  BIL0G3.  This  program  uses  the 
marginal  maximum  likelihood  method  for  item  calibration. 

A  principal  axis  factor  analysis  with  the  screen  plot  for  factor  extraction  as  well  as  a  dimensionality  test 
suggested  by  Bejar  (1980)  provided  evidence  that  the  72-item  pool  met  the  assumption  of 
unidimensionality.  The  3-parameter  model  computer  by  BILOG  seemed  to  have  the  best  fit,  with  only 
3  items  showing  fit  statistics  with  p-values  of  less  than  .01. 

ANOVA  with  Tukey  post  hoc  comparisons  were  used  to  compare  the  parameters  and  fits  for  both  data 
sets.  The  items  selected  for  inclusion  under  various  selection  criteria  are  contrasted  among  three 
parameterization  models. 
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Modeis  for  Using  Computer  Facilitated  Communications 
to  Enhance  Instructional  Interactivity 

Norman  R.  Dodl  D.  Michael  Moore,  and  John  K.  Burton;  Virginia  Tech 

This  presentation  examines  several  models  of  instructional  interactivity  within  a  distance 
education  structure.  Satellite  J  livered  video  and  computer  facilitated  communication  provide 
the  primary  contexts  within  which  interactivity  is  discussed.  Two  Virginia  Department  of 
Education  programs,  the  Electronic  Classroom  and  Virginia  Pen,  provide  examples  for 
explaining  the  models  presented. 

Instructional  interactivity  is  defined  as  verbal  interaction  between  a  teacher  and  one  or  more 
students.  Verbal  interaction  occurring  within  conventional  classrooms  has  been  exhaustively 
studied,  however,  in  the  distance  education  context  the  research  is  far  from  exhaustive.  Because 
face-to-face  interaction  is  not  generally  assumed  as  a  given  within  distance  education,  the  models 
presented  in  this  presentation  provide  useful  springboards  for  much  needed  instructional 
research. 

Models  for  interactivity  are  presented  for  three  instructional  stages:  1)  predelivery;  2)  during 
delivery;  and  3)  post  delivery.  While  the  models  presented  include  such  conventional  means  as 
voice  and  fax  transmission,  the  major  focus  for  this  presentation  is  on  computer  facilitated 
communications.  Satellite  instructional  delivery  and  wide  area  interconnectivity  via  computers 
are  becoming  relatively  common  at  the  higher  education  level  but  are  expanding  rapidly  to  the 
K-12  arena.  The  models  for  enhancing  instructional  interactivity  by  integrating  the  unique 
capabilities  of  these  two  technologies  presented  in  this  presentation  are  offered  in  the  hopes  of 
generating  increased  dialogue  and  research  among  those  interested  in  enhancing  the  interactive 
dimensions  of  the  distance  education  process. 

What  is  the  Place  of  Theory  in  Instructional  Design  and  Educational  Technology? 
A  Discussion  of  the  Implications  for  Research  and  Application 

MichaelJ.  Jacobson,  The  University  of  Illinois  at  Urbana- Champaign; 
Steven  Tripp,  University  of  Kansas; 
M.  David  Merrill,  Utah  State  University; 
David  Jonassen,  University  of  Colorado  at  Denver; 
Cynthia  Leshin,  Maricopa  Community  College; 
Barbara  Grabowski,  The  Pennsylvania  State  University 

This  panel  discussion  considers  the  role  of  theory  in  the  fields  of  instructional  design  and 
educational  technology.  This  is  an  important  topic  given  concerns  in  the  Hterature  about  general 
epistemological  and  philosophical  issues,  lack  of  theoretical  perspectives  in  research  and 
practice,  and  inconsistent  and  poor  quality  research.  Furthermore,  recently  there  has  been  a 
renewed  multidisciplinary  interest  in  the  psychological  and  educational  foundations  of 
technological  learning  environments.  The  potential  exists  for  renewing  and  extending  the 
intellectual  foundations  of  fields  associated  with  the  design  and  use  of  instructional  technologies 
based  on  the  juxtaposition  of  ideas  from  different  disciplinary  paradigms  and  different 
communities  of  scholars,  researchers,  and  practitioners. 
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The  panel  members  will  discuss  important  themes  and  issues  related  to  the  central  role  of  theory 
in  the  design,  research,  and  use  of  instructional  systems.  The  following  topics  will  be  discussed: 
Dr.  M.  David  Merrill,  The  Importance  of  Theory  to  Instructional  Design;  Dr.  Steven  Tripp,  No 
Theory  of  the  Design  of  Complex  Instructional  Artifacts  is  Possible;  Dr.  Michael  J  Jacobson, 
Confronting  a  Pluralistic  Theoretical  Universe:  Instructional  Design  and  Recent  Cognitive 
Theories  of  Learning  and  Instruction;  Dr.  David  H.  Jonassen,  The  Implications  of  Constructivist 
Environments  for  Classroom  Implementation  and  the  Evaluation  of  Learning;  and  Dr.  Cynthia 
Leshin,  The  Application  of  Theory  and  Research  to  Real  World  Situations.  Dr.  Barbara 
Gmbowski  will  serve  as  discussant. 


Automating  Instructional  Design  for  Teaching  Dynamic  Processes 

Mark  K.  Jones 

Computer-based  instruction  is  attractive  for  many  reasons,  but  the  dollar  and  time  costs  of 
authoring  CBI  are  extremely  high.  A  series  of  research  projects  at  Utah  State  University  have 
as  their  goal  the  development  of  methods  and  tools  for  automating  portions  of  the  instructional 
design  process  to  reduce  those  costs.  The  focus  of  the  research  is  on  three  areas:  identifying 
and  representing  domain-independent  categories  subject  matter,  identifying  reusable  instructional 
algorithms  for  these  categories,  and  representing  instructional  design  expertise  for  customizing 
the  algorithms  for  specific  students,  environments,  and  tasks. 

This  presentation  focuses  on  a  single  category  of  human  performance:  the  operation  of  devices. 
Skill  in  device  operation  builds  primarily  upon  skills  in  identifying  and  locating  the  parts  of  the 
device,  comprehending  the  process  which  explain  the  functioning  of  the  device,  and  knowing 
the  steps  of  the  procedure  for  operating  the  device  including  the  conditions  under  which  to 
perform  each  step. 

Subtopics  to  be  covered  include  the  representation  of  the  subject  matter,  including  device  parts, 
procedure  steps,  and  processes,  with  a  focus  on  modelling  the  functioning  of  the  device,  and  the 
reusable  algorithms  for  instructing  device  operation.  It  is  the  author's  belief  that  the  methods 
developed  for  teaching  device  functioning  will  apply  to  other  dynamic  processes.  Examples  of 
the  tools  being  developed  will  be  shown. 


Integrated  Instruction  for  Complex  Computerized  Tools 

Joan  C.  Lee,  US  Army  CERL 

Many  tutorials  have  been  developed  to  teach  novices  to  use  computerized  tools  such  as  word 
processing  and  graphics  software.  After  learning  with  tutorials,  users  experience  little  difficulty 
in  using  the  tools  because  of  previous  similar  experiences  in  a  non -automated  form  (i.e. 
typewriting  and  hand  drawing).  However,  designing  an  instructional  software  for  learning  a 
complex  computerized  tool  such  as  Computer-Aided  Design  and  Drafting  (CADD)  requires 
special  consideration  because  many  CADD  functions  have  no  counterpart  in  the  non- 
computerized world.  Integrated  instruction,  that  which  incorporates  both  domain-specific  and 
experiential  CADD  knowledge,  bridges  the  gap  between  the  CADD  novice  and  the  CADD 
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expert  skill  levels.  The  goal  of  integrating  CADD  instruction  is  to  provide  users  with  the 
knowledge  and  ability  to  solve  real-world  problems,  whether  they  be  simple  or  complex,  familiar 
or  novel. 

We  propose  an  architectural  model  of  integrated  CADD  instruction  where  architects  learn  to 
respond  flexibly  to  a  multitude  of  design  and  drafting  problems  in  the  CADD  environment.  The 
theoretical  framework  for  our  model  is  Rand  Spiro;s  Cognitive  Flexibility  Theory,  which 
emphasized  situation-dependent  knowledge  integration  rather  than  the  retrieval  of  general 
knowledge  from  an  introductory  tutorial. 


Accounting  Instruction  through  Structural  Learning  and  Elaboration  Theories: 

A  Hypertext  Comparison 

Tom  Lightner  and  Eric  Smith,  University  of  Northern  Colorado 

Two  hypertext  programs  to  teach  the  accounting  cycle  to  undergraduate  business  majors  have 
been  developed  with  two  key  proposes:  to  provide  business  schools  with  an  innovative  way  to 
complement  their  current  instructional  methods  and  to  create  a  vehicle  to  test  two  instructional 
design  models. 

The  programs  meet  the  first  goal  by  using  ToolBook's  hypertext  capabilities  to  facilitate  the 
linking  of  related  information.  The  content  comprises  the  basics  of  the  accounting  cycle  or  the 
breadth  of  about  one  third  of  one  semester  of  college  level  accounting  instruction.  The 
information  is  presented  in  small  doses  with  opportunities  for  practice  and  immediate  feedback. 
The  second  goal  was  met  by  designing  each  of  the  programs  with  the  guidelines  of  a  different 
instructional  theory.  One  was  designed  using  elaboration  theory,  while  the  other  followed 
structural  learning  theory. 

The  same  development  team  using  the  same  content  and  development  tools  for  each  theory.  The 
programs  were  used  as  adjunct  instruction  to  the  instructors*  lectures.  This  paper  discusses  the 
research  design  and  results  of  the  pilot  study.  Results  generalize  only  within  this  narrow  scope, 
though  the  authors  will  speculate  on  the  application  of  this  approach  in  other  contexts. 

Can  Graphical  Browsers  Affect  a  Learner's 
Structural  Knowledge  in  a  Hypertext  Environment? 

Eileen  Schroeder,  Penn  State  University 

Traditional  linear  instruction  incorporates  an  expert's  conception  of  the  structure  of  the 
knowledge  through  the  organization  of  the  text,  text  elaboration  techniques,  or  through  graphic 
portrayals  of  the  structure.  Learners  are  thought  to  assimilate  the  new  information  into  their 
own  cognitive  frameworks,  adding  to  existing  structures,  modifying  them,  or  creating  new 
structures^  Hypertext  can  allow  the  designer  or  user  to  link  information  together  and  create 
paths  through  a  body  of  material,  offering  the  learner  control  over  his/her  path  through  the 
instruction  without  many  of  the  traditional  connective  aids. 
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Building  on  the  theoretical  base  of  information  processing  and  schema  theory,  this  study 
examines  a  method  for  facilitating  the  acquisition  of  a  lesson's  factual  and  structural  knowledge 
and  its  integration  into  the  cognitive  frameworks  of  learners  with  different  degrees  of  prior 
knowledge.  It  examines  the  achievement  of  students  with  different  degrees  of  knowledge  in  the 
content  area  using  graphical  browsers  with  the  links  either  labeled  or  unlabeled  portraying  the 
structure  of  the  content  versus  using  embedded  "hot  words"  to  navigate  through  the  system. 
Achievement  will  be  measured  in  terms  of  the  learner's  comprehension,  factual  knowledge,  and 
structural  knowledge  after  using  the  systems. 


The  Instructional  Media  Research  Debate:  Medium  or  Message? 

Theodore  Shlechter,  Army  Research;  /.  Dexter  Fletcher,  Institute  for  Defense  Analysis 

A  reoccurring  debate  in  the  instructional  media  literature  involves  determining  whether  the 
effects  of  any  instructional  medium  are  due  to  qualities  inherent  in  the  particul'^  medium  or  to 
the  instructional  content,  which  may  be  delivered  by  any  media.  Clark  (1983)  has  rekindled  this 
debate  with  regards  to  computer-based  education  (CBE)  by  claiming  that  the  evidence  indicated 
that  such  instructional  media  are  mere  instructional  vehicles.  And,  Dr.  Theodore  Shlechter 
(1990)  has  also  argued  that  conclusive  evidence  regarding  the  effectiveness  of  CBE  does  not 
exist.  Drs.  Dexter  Fletcher  (1991)  and  Robert  Kozma  (1991),  however,  believe  that  media 
effects  do  exist.  Dr.  Kozma  has  argued,  for  example,  that  the  inherent  symbolic  system 
associated  with  computers  makes  this  technology  most  appropriate  for  certain  types  of 
educational  purposes.  Dr.  Eva  Baker  has  advocated  an  alternative  position  to  those  previously 
discussed.  She  has  suggested  that  any  conclusive  statements  regarding  this  debate  require  the 
development  of  new  evaluation  tools  which  are  capable  of  measuring  the  complexities  of  the 
instructional  effects  associated  with  this  medium.  This  panel  discussion  will  thus  allow 
advocates  on  opposing  sides  of  the  instructional  media  research  debate  to  collaborate  in 
articulating  and  resolving  their  differences. 


The  Application  of  Cognitive  Flexibility  and  Constructivism  Theories 
to  a  Hypermedia  Course  on  Thermal  Processing 

Wendy  Snetsinger  and  Susan  Clark,  Penn  State  University 

Computer-based  instruction  delivered  in  a  hypermedia  format  is  art  excellent  environment  to 
engage  learners  by  providing  self-discovery  learning  opportunities.  In  essence,  this  is  ideal  for 
applying  cognitive  flexibility  theory  and  constructivism  to  hypermedia. 

At  Penn  State  University  students  taking  Food  Science  430,  "Thermal  Processing",  enter  from 
a  wide  range  of  academic  disciplines.  None  have  had  courses  in  thermal  processing  -  which 
is  the  theory,  mechanics  and  application  of  principals  used  in  the  preparation  of  food  through 
canning.  This  subject  involves  aspects  of  at  least  three  disciplines  —  chemical  engineering, 
microbiology  and  nutrition. 
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One  of  the  objectives  of  the  course  is  to  harmonize  the  interdisciplinary  information  in  a  way 
that  students  from  diverse  perspectives  can  understand  and  appreciate  the  various  components. 
A  second  objective  is  for  the  students  to  be  able  to  transfer  knowledge  to  new  and  unique 
applications. 

Through  simulations  presented  in  CBI,  students  are  able  to  see  how  thermal  processing  involves 
the  interrelationship  of  different  academic  perspectives.  Further,  the  students  must  make  choices 
as  to  what  variables  they  wish  to  analyze  and  manipulate.  They  discover  immediately  through 
feedback  that  employs  animation,  graphics,  sound  and  text,  what  the  consequences  are  of  those 
choices. 


The  Effects  of  Audio  Resources  on  Learning  from  a  Hypertext  Bilingual  Dictionary 

Steven  Tripp,  University  of  Kansas 

In  an  extension  of  research  presented  in  previous  years,  we  looked  at  the  effects  of  adding 
phonological  information  to  a  hypertext  dictionary.  As  in  previous  experiments,  dependent 
measures  in  this  experiment  are  recognition  and  recall  of  Japanese  vocabulary  after  learning 
from  a  hypertext  dictionary.  We  previously  hypothesized  that  a  lack  of  information  about  the 
semantics  hyperspace  would  cause  disorientation  which  would  in  turn  draw  upon  limited  mental 
resources,  reducing  learning.  We  found  that  disorientation  did  result  in  reduced  learning,  but 
adding  non-congruent  orienting  devices  did  not  necessarily  increase  learning.  In  a  follow-up 
experiment  we  used  congruently  metaphorical  graphics  with  an  advance  organizer.  Results 
supported  the  hypothesis  that  congruency  enhances  retention.  With  the  intention  of  gaining  more 
information  about  learning  from  an  elaborated  hypertext/hypermedia  dictionary,  we  added 
pronunciation  information  to  our  software.  The  use  of  audio  resources  in  computer-based 
instruction  is,  to  the  best  of  our  knowledge,  a  nearly  completely  unexplored  domain.  It  is  our 
hypothesis  that  the  encoding  of  two  sources  of  information  (graphemic  and  phonemic)  should 
allow  better  retrieval  of  vocabulary  due  to  the  greater  number  of  access  routes.  In  general,  the 
audio  channel  is  an  unused  and  available  medium  of  computer  communication  and  its  value 
should  be  explored. 

The  Effects  of  Graphical  Display  Characteristics  in  a  Computer-Based  Environment 

James  B.  Wells,  North  Georgia  College;  Ben  H.  Layne,  Georgia  State  University 

With  the  increasing  sophistication  of  computational  machinery  and  software  packages  offering 
automated  visual  displays  of  quantitative  information,  there  has  been  a  recent  renaissance  of 
statistical  graphics.  To  meet  the  challenge  of  this  renaissance,  some  scholars  have  emphasized 
the  major  need  for  a  thorough  review  of  the  entire  graphical-statistical  methodology  and  tlie 
development  of  a  theory  of  graphical  methods.  To  illustrate,  research  with  graphical  symbols 
indicates  that  some  of  them  are  not  perfectly  perceived  by  the  reader  and  may  suffer  from 
biases.  With  regard  to  these  errors  in  perception  and  biases,  other  problems  and  questions 
emerge.  For  example,  should  readers  be  warned  about  potential  graphical  biases?  Or  should 
the  graphical  symbols  be  altered  and  corrected? 
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The  research  question  this  study  will  attempt  to  answer  pertains  to  an  investigation  of  the 
potential  effects  of  computerized  graphical  display  characteristics  on  the  accuracy  and  bias  of 
estimates  of  whisker  length  of  two  different  types  of  box-and-whisker  plots.  Whereas  previous 
research  has  shown  that  the  regular  (unnotched)  box-and-whisker  plot  demonstrated  some 
inaccuracy  and  bias  in  its  ability  to  display  quantitative  information,  no  research  has  been 
performed  to  demonstrate  the  effectiveness  of  the  notched  box-and-whisker  plot. 


Computer-Based  Research  Formats: 
Utilizing  HyperCard  to  Administer  Research  Projects  and  Collect  Data 

Ronald  Zellner,  Texas  A&M  University 

This  presentation  will  highlight  HyperCard  and  illustrate  how  this  environment  can  be  utilized 
to  design  fully  automated,  Computer-Administrated  Research  formats.  The  design  and  function 
of  data  collection  tools  to  be  used  by  the  researcher  while  observing  research  activities  will  also 
be  illustrated.  Automated,  background  data  collection  which  is  not  part  of  a  specific  research 
procedure  will  be  included.  The  structure  and  function  of  the  HyperTalk  scripting  will  be 
illustrated  in  relation  to  text  and  graphics  functions;  data  collection,  storage  and  display  functions 
will  also  be  covered.  The  presentation  will  highlight  conceptual  and  technical  information  for 
potential  application  to  a  variety  of  educational  research  topics.  Stacks  will  be  demonstrated  to 
present  both  the  products  of  the  applications  (graphics,  data  collection  formats,  data  sets,  etc.) 
and  the  underlying  procedures  (control  scripts,  files  and  subroutines). 

Techniques  for  developing  research  materials  will  be  presented,  including  graphics  scanning, 
scripting,  and  importing  text  and  numerical  data  from  utilities  such  as  word  processors  and 
spread  sheets.  Methods  of  monitoring  and  recording  subject  responses  in  a  variety  of  forms  and 
formats  will  be  illustrated,  including  response  latency  timing,  text  input,  multiple  choice 
responses,  graphic  selection  and  spatial  location.  Techniques  for  data  set  transfer  to  mainframe 
or  microcomputer  packages  (data  base,  spread  sheet,  etc.)  for  analysis  will  be  described. 
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METACOGNITIVE  SKILLS 
IN  COMPUTER-ASSISTED  INSTRUCTION 

Myung-Sun  Clinton  and  James  R.  Okey,  The  University  of  Georgia 


Abstract 

Computer  presentation  of  hypermedia  or  hypertext  is  being  implemented  in  many 
instmctional  settings.  Because  hypertexts  are  arranged  in  a  node-link  fashion 
which  peimits  learners  to  explore  material  in  the  sequence  which  makes  the  most 
sense  to  them  (based  on  their  existing  knowledge  base),  the  importance  of 
metacognitive  control  of  learning  is  critical.  Metacognition  refers  to  an  awareness 
of  one's  cognitive  processes^  of  what  things  are  to  be  done,  and  of  how  to 
accomplish  them. 

Research  on  self-regulation  and  monitoring  skills  of  students  through  the  use  of 
metacognitive  learning  strategies  with  computer-assisted  instmction  has  had 
promising  results.  This  paper  will  review  definitions  of  metacognition  and  research 
findings  related  to  metacognitive  strategies  in  CAI.  Implications  of  findings  in 
metacognition  for  computer-based  instructional  design  and  development  will  be 
provided. 

Introduction 

Given  the  rapidly  changing  environment  of  the  world,  traditional  instmction  no 
longer  prepares  the  learner  to  know  their  responsibilities.  It  does  not  provide  a 
basis  for  adaptive  problem  solving  which  involves  careful  examination  of  personal 
and  organizational  behavior  and  the  development  of  workable  alternatives 
(Amistrong,  1989). 

What  is  Metacognition 

Metacognition  refers  to  an  individual's  ability  to  accurately  determine  the  goal  of  a 
given  task,  apply  appropriate  strategies  to  reach  the  goal,  monitor  progress  toward 
the  goal,  and  adjust  strategies  as  necessary.  Within  the  last  15  years,  metacognition 
has  become  one  of  the  major  fields  of  cognitive  developmental  research. 

Ravell  (1976)  used  a  description  of  learning  activities  to  define  metacognition: 

"  Metacognition  refers  to  one's  knowledge  conceming  one's  own  cognition 
processes  and  products  or  anything  related  to  them,  e.g.,  the  learning-relevant 
properties  of  information  or  data.  For  example,  I  am  engaging  in 
metacognition  (metamemory,  metaleaming,  metattention,  metalanguage,  or 
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whatever)  if  I  notice  that  I  am  having  more  trouble  learning  A  than  B\  if  it 
strikes  me  that  I  should  double-check  C  before  accepting  it  as  a  fact;  if  it  occurs 
to  me  that  I  had  better  scrutinize  each  and  every  alternative  in  any  multiple- 
choice  type  task  situation  before  deciding  which  is  the  best  one;  if  I  become 
aware  that  I  am  not  sure  what  the  experimenter  really  wants  me  to  do;  if  I  sense 

that  I  had  better  make  a  note  of  D  because  I  may  forget  it  Metacognition 

refers,  among  other  things,  to  the  active  monitoring  and  consequent  regulation 
and  orchestration  of  these  processes  in  relation  to  the  cognitive  objects  or  data 
on  which  they  bear^  usually  in  the  service  of  some  concrete  goal  or  objective" 


Metacognition  comprises  two  kinds  of  knowledge.  The  first  kind  of  knowledge 
about  metacognition  is  an  individual's  awareness  of  his  or  her  own  cognitive 
resources  in  relation  to  the  task.  That  is,  the  learner  understand  what  skills, 
strategies,  and  resources  are  needed  to  accomplish  a  task.  These  activities  include 
finding  main  ideas,  paraphrasing  information,  creating  images  of  verbal 
information,  copying  material,  taking  selective  notes,  and  underlining  important 
parts  of  an  article. 

The  other  kind  of  knowledge  about  metacognition  refers  to  the  activities  used  to 
regulate  and  control  learning.  That  is,  learner  knows  how  and  when  to  use  these 
skills  and  strategies  to  make  sure  the  task  is  completed  successfully.  These 
processes  include  planning  activities  (predicting  outcomes,  scheduling  strategies, 
and  various  forms  of  trial  and  error);  monitoring  activities  (monitoring,  testing, 
revising,  and  re-scheduling  one's  strategies  for  learning);  and  checking  outcomes 
(evaluating  the  effectiveness  of  one's  efforts)  (Brown,  1987). 

Metacognitive  Awareness 

A  leamer*s  metacognitive  awareness  is  influenced  by  learner,  task,  and  strategy 
variables.  Flavell  and  Wellman  (1977)  hypothesized  that  learners  continually  gain 
additional  knowledge  about  how  leamer,  strategy,  and  task  variables  affect  both  the 
quality  and  quantity  of  learning.  Leamers  have  die  opportunity  to  learn  about 
themselves  with  each  new  academic  environment.  For  example,  they  learn  about 
their  differential  effectiveness  on  tasks  (e.g.,  I  am  better  at  arithmetic  than  at 
spelling). 

Task,  strategy,  and  leamer  variables  interact  when  leamers  engage  in  metacognitive 
activities.  Leamers  consider  the  type  and  length  of  material  to  be  leamed  (task),  the 
potential  strategies  to  be  used  (strategy),  and  their  skill  at  using  the  various 
strategies  (leamer).  For  example,  leamers  might  believe  that  they  should  take  notes 
rather  than  underline  in  finding  the  main  ideas  of  an  article. 


(p.  232). 
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Metacognitive  Experiences 

Metacognitive  experiences  consist  of  self-regulation  and  monitoring  activities. 
These  activities  are  observable  aspects  of  metacognition  and  presuppose  the 
existence  of  knowledge,  because  it  is  assumed  that  a  strategic  activity  is  based  on 
existing  knowledge. 

Metacognitive  experiences  also  can  activate  two  types  of  strategies  which  are 
cognitive  and  metacognitive.  Forrest-Pressley  and  Waller  (1984)  have 
dintinguished  that  cognition  refers  to  the  actual  processes  and  strategies  that  are 
used  by  tlie  leamer  and  metacognition  is  a  construct  that  refers,  first,  to  what  a 
person  knows  about  his  or  her  cognitions  and  second,  to  the  ability  to  control  these 
cognitions  (p.  6),  Meichenbaum  (1985)  also  referred  to  cognition  as  the  use  of 
operations  in  leaming,  and  described  metacognition  as  the  learner's  overseeing  of 
these  cognitive  operations.  That  is,  cognitive  strategies  are  involved  in  using 
cognitive  processes  and  metacognitive  strategies  in  choosing  cognitive  processes. 

Metacognitive  learning  strategies  are  not  intended  to  teach  leamers  specific  course 
content.  Rather,  these  are  generalizable,  transferable  cognitive  skills  designed  to 
help  leamers  control  their  leaming.  Higher  achievers  are  efficient  leamers  because 
they  know  what  they  do  and  do  not  know,  which  enables  them  to  fill  information 
gaps  when  they  exist.  This  monitoring  of  comprehension  is  designed  to  improve 
self-control  and  self-regulation  of  the  leaming  process.  Flavell  (1981)  argues  that 
cognitive  monitoring  is  a  necessary  step  before  there  can  be  any  integration  and 
transfer  of  knowledge,  and  that  cognitive  monitojring  is  the  key  to  critical  thinking. 
Zimmerman  and  Pons  (1986)  have  explored  extensively  the  differences  in  this 
self -regulation  component  of  metacognition  in  high-  and  low-achieving  high  school 
students.  Based  on  stmctured  interviews  designed  for  both  academic  and  social 
contexts,  they  concluded  that  skilled  leamers  used  more  strategies  consistently 
across  contexts  than  did  less  skilled  leamers. 

Computer- Assisted  Instruction  (CAI)  and  Metacognition: 
Related  Research  Findings 

The  computer  has  been  modeled  by  many  researchers  on  human  cognitive  process 
systems.  Hartly  (1980)  recommends  that  students  try  solutions  in  a  computer 
program  format  because  it  requires  that  the  student  give  a  precise  statement  in  a 
step-by-step  solution.  Flavell's  (1981)  model  of  cognitive  monitoring  resembles 
the  computer  model  because  it  requires  describing  the  probbm  or  issue,  estimating 
the  outcome  or  end  product,  deciding  upon  a  strategy  and  implementing  it, 
monitoring  the  outcome,  and  evaluating  the  process. 
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Pea  (1985)  has  called  for  more  research  into  the  effects  of  cognitive  process 
systems,  i.e.,  systems  that  externalize  the  products  of  the  mind  and  leave  a  path 
showing  where  one  has  been  in  an  episode  of  problem  solving.  The  computer's 
capacities  represent  practical  means  of  testing  whether  cognitive  process  systems 
will  make  developmental  contributions  to  human  reasoning. 

Clements  and  GuUo  (1984)  have  studied  the  effect  of  a  CAI  environment  on 
metacognitive  skills.  They  provided  12  weeks  of  instraction  to  18  first-grade 
children,  randomly  assigned  to  either  a  Logo  or  a  Computer-Assisted  Instruction 
(CAI)  group.  Posttesting  revealed  that  the  Logo  programming  group  significantly 
outperformed  the  CAI  group  on  both  metaf  ognition  tasks.  The  ability  to  monitor 
one's  own  thinking  and  to  realize  when  one  does  not  understand  may  be  positively 
affected  by  computer  programming  environments  in  which  problems  and  solution 
processes  are  brought  to  an  explicit  level  of  awareness  and  in  which  consequent 
modification  of  problem  solutions  is  emphasized.  Through  consistent  feedback  in 
the  form  of  a  visual  representation  of  the  procedures  and  sequences  of  their  own 
thinking  processes,  children  may  have  learned  how  to  monitor  those  processes. 

In  a  follow-up  study  (Clements,  1986),  as  in  the  first  study  (Clements  &  Gullo, 
1984),  measures  of  creativity,  classification,  sedation,  metacognitive  abilities,  and 
achievement  were  used.  Seventy-two  first-  and  third-grade  children  were 
randomly  assigned  to  one  of  three  conditions:  Logo,  CAI,  and  control.  After  22 
weeks  of  computer  experience,  both  the  first  and  third  graders  in  the  Logo 
treatment  outscored  the  CAI  and  control  children  on  measures  of  operational 
competence,  two  of  three  measures  of  metacognition  skills,  and  a  measure  of 
creativity  and  on  describing  directions.  No  differences  were  found  on  measures  of 
reading  and  mathematics  achievement. 

For  the  replication  study  of  Clements  (1986)  and  Markman  (1977),  Lehrer  and 
Randle  (1987)  investigated  an  experimental  study  of  the  instmctional  effectiveness 
of  two  types  of  interactive  software  environments.  Subjects  were  thirty-nine 
first-grade  children  enrolled  in  a  New  York  City  school  serving  a  low  SES 
population.  An  alternate  ranks  design  was  employed  to  assign  subjects  to  one  of 
three  treatment  conditions:  Control,  Logo  programming,  and  Interactive  Software. 
Students  in  the  Logo  and  Interactive  Software  groups  were  instmcted  for  five 
months,  two  times  per  week  for  approximately  twenty  to  twenty-five  minutes  each 
time.  Students  in  the  Control  group  were  instmcted  once  each  week  for  forty-five 
minutes.  At  the  end  of  instruction,  measures  of  problem  solving,  metacognition, 
and  composition  were  administered  to  all  students.  The  measure  of  problem 
representation  was  identical  to  that  of  Clements  (1986).  The  directions  and 
materials  of  the  comprehension  monitoring  task  were  identical  to  those  proposed  by 
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Markman  (1977).  Results  showed  that  Logo  and  Interactive  Software 
environments  enhanced  problem  solving  performance  for  a  novel  task,  but  the 
Logo  environment  was  most  facilitative  for  "learning  to  learn."  However,  both 
environments  appeared  to  enhance  components  of  metacognition:  comprehension 
monitoring  and  tiie  ability  to  monitor  and  to  integrate  old  and  new  information. 

The  development  and  enhancement  of  self-regulation  via  computer  has  been 
predicted  on  the  ability  to  access  and  engage  one's  metacognitive  skills  (Armstrong, 
1989).  Fifty-two  students,  27  boys  and  25  girls  in  grades  3,  5,  and  8,  participated 
in  a  study.  It  consisted  of  nine  50-minute  sessions  over  a  two-week  period,  with 
approximately  ten  students  in  each  session.  In  the  initial  session,  the  entire  group 
was  introduced  to  ten  things  a  computer  can  do.  In  the  second  through  ninth 
sessions,  the  students  were  given  a  choice  of  completing  specific  tasks  individually 
or  with  partners.  The  protocol  sheets  were  designed  to  include  tlie  problem- 
solving  steps  which  are  comparable  to  the  metacognitive  skills  of  planning,  error 
monitoring,  reality  testing,  and  re-evaluation. 

The  computer  became  the  receiver  of  the  students'  inputs  and  strategies  and  served 
as  a  model  and  structure  tlirough  which  to  gain  access  to  their  own  personal 
problem-solving  strategies.  Students  analyzed,  criticized,  adapted,  and  changed 
those  strategies  as  needed.  Nonproductive  steps  (or  error  type),  tasks  attempted, 
tasks  completed,  task  sequences,  and  the  debugging  and  monitoring  of  strategies 
employed  were  recorded.  These  data  were  analyzed  by  sex,  grade  level,  and  task 
type.  Proficiency  was  found  to  have  increased  and  strategies  were  more  easily 
adopted  and  adapted  to  other  tasks  and  problems.  Completion  of  tasks  increased 
significantly  and  nonproductive  steps  (errors)  and  number  of  trials  (redos) 
decreased.  There  were  no  differences  by  sex  except  in  the  types  of  errors  made. 
Evidence  of  self -regulation  development  was  also  shown  in  the  types  of 
questioning  used  by  students. 

Mikulecky,  Clark,  and  Adams  (1989)  also  found  that  significant  differences  in 
examination  results  supported  the  hypothesis  that  "how  to"  reading  strategies  could 
be  taught  via  computer  and  that  college  students  were  then  able  to  transfer  the 
st^tegies  to  study  new  material.  Moreover,  interview  and  questionnaire  data 
indicated  that  students  view  computer-assisted  instmction  positively  as  a  way  to 
learn  strategies  for  reading  difficult  material. 

Implications 

As  shown  in  the  literature  review,  instruction  that  provides  conditional  knowledge 
(knowing  "when"  and  "why"),  self-regulation,  and  monitoring  strategies  has 
several  positive  effects  on  die  leamer*s  performance  in  computer-assisted 
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instruction.  Schmitt  and  Newby  (1986)  advocate  that  learning  conditional 
knowledge  can  be  incorporated  with  traditional  systematic  designs  of  instruction. 
Such  systematic  instructional  design  would  include  the  following:  gaining 
attention;  presenting  information  of  the  "what"  (declarative  knowledge)  and/or 
"how"  (procedural  knowledge)  of  the  skill  to  be  leamed;  and  enhancing 
information  through  practice,  feedback,  and  follow-up  activities.  Paris  and 
colleagues  (1983)  suggest  that  to  use  a  new  strategy  properly  the  learner  should  be 
informed  of  the  importance  of  the  strategy  ("why")  during  the  gaining-attention 
phase.  Baumann  and  Schmitt  (1986)  suggest  that  the  learner  should  also  be 
informed  of  "when"  the  task  is  relevant  after  the  presentation  of  the  "what"  and/or 
"how"  of  the  strategy. 

Since  many  learners  do  not  use  effective  strategies,  strategy  instruction  seems  to  be 
beneficial  to  leamers.  The  ultimate  goal  in  CAI  is  to  have  learners  know  how  to  use 
strategies  independently,  skillfully,  and  appropriately.  Therefore,  metacognitive 
strategies  should  be  taught  to  leamers  directly  over  an  extended  period  of  time  as 
part  of  the  existing  CAI.  That  is,  leamers  should  be  provided  input  about  when 
and  where  to  use  the  strategies  they  are  learning,  and  they  should  be  shown  how 
their  performance  improves  by  using  the  right  stiategies. 

To  summarize,  metacognition  comprises  awareness  and  experiences  of  cognitive 
processes.  Instructional  design  for  conditional  knowledge  ("when"  and  "why") 
can  be  incorporated  in  traditional  instructional  design.  The  first  step  in  teaching  a 
strategy  is  to  describe  the  strategy  to  students.  This  can  be  accomplished  by  the 
teacher  modeling  actual  use  of  strategy,  particularly  with  "think  aloud"  techniques 
about  how  to  execute  the  procedure.  In  addition,  strategy  instmction  should  be 
explicit,  intensive,  and  extensive. 
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The  Instructional  Design  Taxonomy:  Further  Refinements  for  Describing  and  Measuring  the 

Interactivity  of  Compuler-Based  Instruction 

Dale  Y,  Hoskisson 
Division  of  Education  of  Education  and  Psychology 

Abstract 

This  paper  describes  two  components  of  a  taxonomy  for  describing  the  dynamic  quality  of 
instructional  systems  independent  of  media.  The  first  component  includes  terms  for  describing  the 
feedback  pattern  of  an  instructional  system  in  responding  to  a  student  error.  The  second  component  is 
an  application  of  the  terms  to  develop  an  instructional  flowchart  that  creates  a  visual  picture  of  the 
interactive  qualities  of  the  system. 

Introduction 

This  paper  describes  certain  aspects  of  a  proposed  taxonomy  (Hoskisson  1991,  1992a,  1992b)  being 
developed  at  the  Center  for  Systematic  Courseware  Integration,  Design,  and  Evaluation  (C-SCIDE)  that 
we  hope  will  accomplish  two  things.  The  first  concerns  educational  research  that  compares  two  types  of 
media  or  one  theory  of  instruction  to  another.  We  have  not  been  successful  in  showing  any  significant 
difference  in  this  endeavor  (Heinich,  Molenda,  &  Russell,  1989).  One  of  the  reasons  is  that  we  are  not 
asking  why  a  particular  systems  works  well  or  is  better  than  another  (Brandt,  1989).  I  believe  tliat  we 
need  a  way  to  describe  instruction  without  reference  to  media  or  theory  before  we  can  successfully 
compai'e  instructional  systems. 

The  other  concern  deals  with  the  power  of  the  computer.  One  of  the  important  characteristics  of  the 
computer  is  its  ability  to  manipulate  information  more  quickly  and  efficiently  than  we  humans.  It  is  this 
manipulative  power  that  makes  the  computer  such  a  valuable  tool  in  education.  This  ability  makes  it 
possible  to  truly  individualize  instmction.  To  determine  the  value  of  a  CBT  program  we  need  a  way  to 
describe  tlie  degree  to  which  the  program  takes  advantage  of  this  dynamic  power  to  manipulate 
information. 

An  important  aspect  in  determining  the  dynamic  quality  of  an  insti-uctional  system  is  how  it  I'esponds 
to  incorrect  answers  from  the  student.  The  feedback  pattern  of  any  system  can  be  described  in  simple 
terms.  The  following  elements  of  the  taxonomy  are  based  on  the  work  of  Brandl  (1989). 

Attempts  allowed:  This  refers  to  how  many  times  the  student  is  allowed  to  aiiempt  to  answer  a 
question  before  being  moved  to  the  next  question. 

Correct  answer  (ca):  Does  the  system  provide  the  correct  answer  before  going  to  the  next  question? 

Reteach  (rt):  Does  the  system  reteach  the  information  and  if  so  is  it  retaught  using  the  same  material 
(rt.s),  new  or  alternate  versions  of  the  original  material  (rt.n),  or  additional  material  (rt.a). 

Hints  or  clues  (h/c):  There  are  many  variations  of  hints  and  clues.  Mathemagenic  aids  can  be  added 
to  the  information.  The  system  can  point  out  or  ask  questions  about  specific  areas  to  guide  the  students 
thinking. 

Error  specific  response  (sr):  How  does  the  system  decide  what  type  of  pattern  to  use?  For  example, 
when  a  student  chooses  the  wrong  answer  to  a  multiple  choice  question,  is  the  system  response 
determined  by  the  specific  option  the  student  chose  or  does  it  use  the  same  pattern  for  each  choice? 


443 


2 

These  elements  can  be  combined  in  an  infinite  number  of  patterns.  Each  pattern  can  be  shown  in  a 
flowchart  style  representation  of  the  instructional  activity.  A  typical  pattern  is  shown  in  Figure  1.  The 
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Figure  1.  Atypical  feedback  pattern. 

system  poses  a  question  and  the  student  responds.  If  the  student  is  correct  the  system  continues  to  the 
next  question.  If  the  student  is  wrong  the  system  indicates  the  answer  is  wrong  and  asks  the  question 
again.  If  the  student  continues  to  answer  incorrectly  the  pattern  is  repeated  x  number  of  times  until  the 
system  gives  up  and  tells  the  student  the  correct  answer.  Figures  2-4  show  more  possible  feedback 
patterns. 

The  pattern  in  Figure  2  shows  that  a  question  is  presented  and  the  student  is  informed  that  he  or  she 
has  answered  incorrectly.  The  student  is  ten  retaught  with  a  new  version  of  the  original  material  and 


Figure  J.  Feedback  pattern  a 


given  another  question.  Figure  3  shows  a  similar  pattern  only  this  time  the  reteaching  is  keyed  to  the 
specific  incorrect  response  of  the  student.  Again  a  new  question  is  asked  and  followed  by  more 
information.  The  pattern  in  Figure  4  is  also  similar  but  returns  the  student  to  the  original  question  after 
reteaching. 

This  type  of  instructional  fiowchait  allows  us  to  compare  instructional  systems  without  reference  to 
the  media  involved  or  to  any  theory.  For  example,  Figures  5  and  6  show  the  flowchart  for  two  systems, 
one  is  a  lesson  from  a  basal  reader  and  the  other  is  a  CBT  program.  It  is  difficult  to  decide  which  is  which 
when  all  we  have  to  look  at  is  how  the  teaching  is  structured  devoid  of  theoretical  color  and  media  glitz. 
We  also  have  a  clear  picture  of  how  interactive  a  program  is.  The  basal  reader  is  as  interactive  as  the  CBT 
program.  They  have  ihe  same  basic  feedback  pattern.  They  pose  a  question,  allow  x  number  of  attempts 
then  give  the  correct  answer,  in  our  work  at  C-SCIDE,  we  have  found  this  to  be  a  typical  feedback  pattern 
for  commercial  educational  materials  of  all  kinds  including  CBT  and  interactive  videodisc  programs. 
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Figure  3.  Feedback  pattern  b 


Figure  4.  Feedback  pattern  c 
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Figure  6.  An  instructional  flowcfsart  for  a  CBT  program 
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in  Computer-Based  Interactive  Courseware 
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Senior  Engineer,  Institute  For  Information  Industry,  Taiwan,  R.O.C. 

(Paper  presented  at  the  34th  ADCIS/SIGCUE  ACM  International  Conference  of 
the  Association  for  the  Development  of  Computer-Based  Instructional  Systems  (ADCIS)  and 
the  Computer  Uses  in  Education  (SIGCUE)  of  the  Association  for  Computing  Machinery 
(ACM).  Norfolk,  Virginia,  U.S.A.,  November  8-11,  1992.) 

Abstract 

This  paper  has  identified  instances  of  learners'  mental  models  (understanding  of 
the  interface)  of  an  interactive  videodisc  training  course  using  a  hypertext  format 
and  a  WIMP  (Window,  Icon,  Menu,  Pointing  device)  interface.   Further,  the  study 
explored  the  interrelationships  among  the  users'  individual  differences  with  respect 
to  learning  styles,  previous  experience  with  computers,  their  mental  models,  their 
content  exposure,  and  their  performance  in  a  computer-based,  interactive  video 
learning  environment.  Subjects,  Instruments,  and  research  procedures  are 
described.  Statistical  analysis  procedures  included  linear  and  multiple  regression 
analyses  of  the  user's  performance  scores,  individual  differences,  mental  models, 
and  content  exposure.  The  implications  of  the  findings  from  both  qualitative  and 
quantitative  perspectives  were  addressed  to  related  research  and  courseware 
interface  design  issues.  Recommendations  for  further  research  v/ere  made. 

Background 

As  the  utilization  of  computers  by  naive  end-users  has  become  widespread,  increasing 
attention  and  effort  have  been  placed  on  producing  more  "user-friendly"  hardware  and 
software.  A  well-designed  user-friendly  computer-based  system  permits  users  to  experience 
mastery  of  the  system,  ease  of  using  it,  competence  in  performance  of  practical  tasks, 
enjoyment,  and  even  eagerness  to  show  it  off  to  others.  Many  of  us  experience  precisely  these 
reactions  with  respect  to  another  major  technology  in  our  lives,  namely,  our  automobiles. 
Regrettably,  experiences  of  frustration  or  anxiety  are  still  common  among  many  computer  users 
(as  well  as  among  some  drivers).  Bailey  (1983)  said  that  in  the  past,  program  designers 
generally  seemed  to  assume  that  users  would  soon  be  transferred,  promoted,  or  quit;  thus  they 
felt  free  to  design  programs  from  their  own  viewpoint  rather  than  that  of  users.  Carroll  (1990) 
maintains  that  even  in  this  age  of  "user  friendly"  software,  many,  if  not  most,  computer  users 
work  at  varying  levels  of  incompetence  rather  than  competence.  Computer  users  in  fields  as 
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diverse  as  banking,  healthcare,  and  communications  struggle  to  understand  the  structure  and 
functions  provided  by  various  application  programs  and  to  learn  the  correct  operating 
procedures  for  programs  that  are  supposed  to  help  them  accomplish  important  tasks.  While  the 
difficulty  involved  in  learning  how  to  use  word-processing,  spreadsheet,  or  telecommunications 
programs  is  considerable,  any  confusion  involved  in  using  programs  designed  for  education  and 
training  is  especially  detrimental.  After  aU,  intei-active  courseware  are  designed  to  help  people 
to  learn,  not  bewilder  them. 

This  problem  can  be  especially  acute  when  interactive  courseware  are  designed  as 
hypertext  environments  that  allow  learners  to  explore  the  content  and  activities  in  the  program 
according  to  their  own  needs  and  interests.  According  to  Shneiderman  and  Kearsley  (1989), 
hypertext  "is  a  database  that  has  active  cross-references  and  allows  the  reader  to  'jump*  to  other 
parts  of  the  database  as  desired"  (p.  3).  Hypertext,  hypermedia,  and  multimedia  are  all  being 
designed  and  used  for  instructional  purposes;  in  each  case,  the  materials  allow  learners  to 
navigate  through  the  contents  and  activities  via  largely  unrestrained  exploration.  However, 
there  is  a  price  for  the  freedom  these  types  of  systems  permit.  Users  of  hypertext  materials  may 
become  confused  and  quickly  lose  track  of  what  is  going  on,  what  they  can  do,  and/or  where 
they  are  located  in  the  program.  The  resulting  frustration  prohibits  learners  from  taking  full 
advantage  of  the  learning  opportunities  presented  by  these  types  of  interactive  courseware. 

Interactivity  and  Exploration 

The  major  advantage  of  interactive  learning  materials  over  other  instructional  media  lies 
in  the  unique  feature  of  "interactivity."  To  be  interactive,  the  exchange  of  information, 
responses  and  feedback  between  the  user  and  the  computer  should  be  as  individualized, 
adaptive,  and  personal  as  possible  (Price,  1991).  A  ideal  form  of  interactivity  would 
approximate  the  exchanges  that  occur  between  a  human  tutor  and  a  student.  The  computer 
makes  the  idea  of  learning  interactively  possible  by  allowing  meaningful  communication 
between  the  learner  and  the  learning  materials.  Maximizing  the  effectiveness  and  efficiency  of 
learner-computer  communication,  i,e,,  interactivity,  leads  to  increased  learner  satisfaction  and 
to  enhanced  performance  and  productivity. 

Interfaces  are  the  communication  vehicles  for  user-computer  program  interaction. 
Various  types  of  interfaces  have  been  created  between  users  and  computer-based  systems  to 
serve  as  bridges  within  the  interactive  process.  In  the  eariiest  days  of  computing,  print-outs 
were  the  major  means  of  perceiving  the  opportunities  for  and  results  of  interactivity,  but  today 
the  computer  screen  is  the  most  common  vehicle  for  perceiving  interactivity.  When  users  view 
a  screen  and  respond  to  computer  systems,  different  interaction  patterns  evolve  depending 
upon  their  unique  cognitive  processes  and  individual  physical  activities. 
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Courseware  that  employs  a  hypertext  interface  encourages  learners  to  explore  content  and 
activities  at  their  own  discretion.  The  hypertext  interface  seems  to  be  particularly  powerful 
because  humans  freely  explore  their  environments  in  everyday  life  for  several  reasons.  The 
stimulation  we  receive  when  exploring  is  one  of  the  primary  motivations  for  human  activity.  It 
is  fun  because  it  fosters  our  curiosity  to  know  more  about  phenomena,  it  results  in  direct 
interaction  with  phenomena,  and  it  provides  us  with  continuous  challenge. 

Cognitive  Load  and  Mental  Models 

Working  with  a  computer  program  requires  different  mental  efforts  than  performing  the 
same  learning  task  via  print  or  by  other  formats,  e.g.,  watching  linear  video.  In  order  to  make 
any  meaningful  response  to  computer  courseware,  the  learner  must  cope  with  and  integrate 
three  cognitive  loads  or  demands,  i.e.,  (a)  the  content  of  the  information^  (b)  the  structure  of 
the  program,  and  (c)  the  response  strategies  available.  These  threefold  demands  imply  that 
there  are  physical,  perceptual,  and  conceptual  contacts  happening  during  learner-courseware 
interactions.  Users  acquire  and  structure  information  delivered  via  interfaces,  conduct  mental 
operations,  and  accomplish  physical  activities  during  their  interactions  with  computer  programs. 
The  limited  capacity  of  working  memory  to  hold  only  five  to  nine  chunks  of  information 
simultaneously  (Miller,  1956)  makes  it  difficult  for  users  of  complexly  structured  courseware  to 
reason  when  numerous  cognitive  load  factors  have  to  be  handled  at  once.  Users  easily  feel 
overwhelmed  by  abundant  menus,  option  commands,  icons,  windows,  functions,  sequence 
controls,  etc.  (Robert,  1989).  This  complexity  increases  the  cognitive  load.  It  also  increases  the 
risks  of  confusion,  especially  when  users  confront  hypertext  materials  which  by  their  very  nature 
include  more  interactive  options  than  many  other  types  of  programs.  The  possibility  of  user 
disorientation  is  a  major  concern  in  the  popular  hypertext  materials  that  feature  a  network-like, 
more  flexible  and  complex  structure  (Byles,  1988;  Conklin,  1987;  Edwards  &  Hardman,  1989; 
Kuhn  &  Streitz,  1989;  Tsai,  1988/89). 

"Mental  model"  theory  is  an  attempt  to  model  and  explain  human  understanding  of 
objects  and  phenomenon  (Gentner  &  Stevens,  1983;  Johnson-Laird,  1983).  The  construction  of 
a  plausible  mental  model  "eases  the  interpretation  and  memorability  of  information"  (Gardiner 
&  Christie,  1987) »  Since  our  understanding  of  human  perception  does  (or  should)  play  a  crucial 
role  in  the  design  of  interfaces,  research  on  mental  models  is  a  promising  approach  for 
analyzing  the  human-computer  interaction  and  eventually  improving  interface  designs. 

Users  seek  new  information  in  ways  that  depend  on  and  are  limited  by  their  current 
mental  model  and  learning  goals  (Tweney,  1987).  The  existence  and  value  of  the  mental 
models  concept  (Brehmer,  1987;  Moray,  1987;  Rasmussen,  1990)  can  be  summarized  by  the 
premise  that  the  quality  of  interaction  within  system  operation  depends  upon  the  functionality 
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of  the  mental  models  users  have  of  the  system.  Mental  models  contribute  to  the  understanding 
of  overall  system  configurations,  their  elements,  and  their  functional  interrelationships. 

The  risk  of  learner  disorientation  in  hypertext  or  other  complex  computer-based  materials 
is  decreased  by  the  quality  of  the  mental  model  of  the  interactive  program  possessed  by  the 
learner.  Users  acquire  knowledge  about  a  computer  program  and  how  to  use  it  through 
experience,  by  instruction,  and/or  by  exploration.  The  better  we  understand  the  ways  users 
interact  with  computer  programs,  the  more  we  can  develop  effective  and  efficient  programs. 
Research  on  mental  models  in  human-computer  interaction  can  identify  salient  characteristics 
of  cognitive  processes  and  help  in  the  development  of  research -based  guidelines  for  the  design 
of  effective  courseware,  thus  linking  the  technology  and  instruction  fields  together.  Hence, 
there  is  a  need  to  identify  the  user's  understanding  about  interfaces  in  terms  of  mental  models 
to  provide  principles  and/or  guidelines  for  computer-based  courseware  design. 

The  Study 

The  overall  purpose  of  this  study  is  to  contribute  to  the  understanding  of  how  people  learn 
in  hypertext-based  interactive  courseware  with  respect  to  users'  understandings  and  exposure  of 
the  courseware.   There  is  little  or  no  strong  theoretical  support  for  design  of  this  type  of 
courseware,  and  therefore  this  is  largely  an  exploratory  study.  Specifically,  this  study  utilized 
correlational  methods  for  gaining  insight  into  the  relationship  among  the  users'  individual 
differences,  their  mental  models  of  the  structure  of  WIMP  (window,  icon,  menu,  and  pointing 
devices)  interfaces  in  interactive  courseware,  their  actual  navigation  activities  during  use  of 
interactive  courseware,  and  the  results  of  that  use.  Analysis  procedures  included  multiple 
regression  analysis  of  the  user's  performance  scores,  individual  differences,  mental  models,  and 
content  exposure.  Qualitative  analysis  based  on  users'  expression  of  the  interfaces  and  users' 
explanations  of  the  specific  content  exposure  were  used  to  categorize  the  instances  of  users' 
mental  models  of  the  course  and  interaction  errors. 

Five  major  questions  were  explored  in  this  study: 
(1)  What  are  the  idea  clusters  and  instances  of  the  mental  models  that  users  form  of  the  WIMP 

I interface  in  hypertext  interactive  courseware  and  how  frequently  do  they  occur? 
(2)  What  individual  difference  in  selected  learning  styles,  interfaces  preferences,  and  previous 
computer  experience  are  related  to  differences  in  users'  mental  models? 
(3)  How  are  differences  in  users'  learning  styles,  interfaces  preferences,  and  previous  computer 
experience  related  to  users'  content  exposure? 
(4)  How  are  different  levels  of  users'  mental  models  related  to  users'  content  exposure? 
(5)  How  are  the  individual  differences,  mental  models,  and/or  content  exposure  of  users 
related  to  the  prediction  of  performance  scores? 
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Subjects,  Instruments,  and  Procedures 

The  design  of  this  study  included  observational  and  correlational  methods.  These 
methods  were  selected  since  no  manipulation  of  a  treatment  variable  was  appropriate 
(Tuckman,  1988)  and  because  of  the  exploratory  nature  of  the  research.  Raw  data  were  treated 
as  continuous  variables  in  this  study.  Linear  and  multiple  regression  methods  were  used  to 
explore  relationships  between  the  specified  variables  and  to  determine  the  extent  to  which  a 
criterion  variable  could  be  predicted.  The  criterion  variable  was  the  performance  scores  of 
users  of  interactive  courseware  that  employed  a  WIMP  interface  within  a  hypermedia 
instructional  treatment.  The  three  possible  predictor  variables  examined  were  individual 
differences  among  learners,  their  mental  models  of  the  courseware,  and  their  content  exposure 
while  using  the  courseware. 

The  sixty  five  participants  for  this  study  were  recruited  as  volunteer  subjects  in  the 
Summer  of  1991.  The  sample  group  in  this  study  included  36  females  and  29  males.  Subjects 
ranged  in  age  from  20  to  57  years  of  age  with  a  mean  age  of  33.06  years.  The  computer  systems 
used  in  this  experiment  were  the  Apple  Macintosh  II  and  Macintosh  SE  personal  computers 
that  use  a  non-traditional  user  interface,  including  features  such  as  window  direct  manipulation, 
icons,  pull-down  menus,  and  a  one-button  mouse  as  a  pointing  device.  This  type  of  interface  is 
commonly  referred  to  as  a  WIMP  interface  for  Windows,  Icons,  Menus,  and  Pointing  devices. 
The  interactive  courseware  used  in  the  study  was  t^^^  "Macintosh  Fundamentals"  (MacFun) 
course  (Apple  Computer,  Inc.,  1990).  MacFun  is  an  award-winning,  interactive  videodisc,  self- 
paced  training  course  delivered  on  Macintosh  computers  interfaced  with  Pioneer  videodisc 
players.  MacFun  was  authored  using  HyperCard  V2.1.  Students  view  text  and  graphics 
materials  on  the  Macintosh  monitor  and  video  on  a  separate  monitor.  Audio  emanating  from 
both  the  videodisc  and  the  HyperCard  stacks  is  heard  through  a  headset.  The  course  was 
developed  by  Training  Support,  Apple  Computer,  Inc.,  for  training  its  own  employees  and 
authorized  Apple  sales  personnel.  The  stated  goal  of  this  course  is  to  help  users  become  self- 
sufficient,  well-rounded  users  of  the  Macintosh  computer.  The  course  which  features  extensive 
technical  content  and  assessment  tests  provides  as  many  as  thirty  hours  of  interactive  training. 
Over  ten  thousand  trainees  have  already  completed  the  course.  This  courseware  was  selected 
because  its  user  interface  had  been  evaluated  extensively  and  revised  several  times  (Gustafson, 
Reeves,  &  Laffey,  1990).  Version  2.0  of  the  MacFun  course  was  used  in  this  study.  MacFun 
includes  twelve  content  modules  and  an  Orientation  module.  Learners  proceed  through  the 
modules  in  whatever  order  they  wish.  The  "Desktop  Operations"  module  was  the  target  module 
used  for  tracking  leainer  interactions  and  performance  scores  in  this  study.  This  target  module 
was  modified  by  the  researcher  by  adding  on-line  pop-up  questions  for  certain  activities 
initiated  by  the  learners. 
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Individual  differences  measures  were  obtained  from  a  PreCourse  Questionnaire  and  the 
Group  Embedded  Figures  Test  (GEFT).  The  PreCourse  Questionnaire  queried  subjects  about 
their  personal  background,  previous  computer  experience,  and  personal  attitudes  toward  and 
icnowledge  of  windows,  icons,  menu,  and  pointing  device  (WIMP)  interfaces.  After  completing 
the  PreCourse  Questionnaire  and  taking  the  GEFT,  each  subject  navigated  through  the 
MacFun  course  individually  at  his/her  own  pace.  MacFun  consists  of  twelve  content  modules 
named  1)  Computer  Basics,  2)  Introduction  to  Applications,  3)  Macintosh  Families,  4)  Desktop 
Operations,  5)  Maintenance,  6)  HyperCard,  7)  Peripheral,  8)  Printers,  9)  Storage,  10) 
Telecommunications,  11)  Networking,  and  12)File   rsnsfer.  An  Orientation  module  serves  as 
a  front-end  tutorial  for  the  MacFun  course.  The  objectives  of  each  module,  the  structure  and 
contents  of  each  section,  and  the  functions  of  various  interface  options  are  explained  in  the 
Orientation  module.  Each  subject  was  requested  to  complete  the  target  Desktop  Operations 
module  as  the  minimum  requirement,  but  each  was  free  to  explore  any  other  modules  he/she 
wished  before  entering  the  target  module.  Desktop  Operations. 

Upon  completion  of  whatever  sections  of  the  MacFun  course  he/she  wished  and  before 
entering  the  target  Desktop  Operations  module,  each  subject  completed  the  PreModule 
Questionnaire  which  was  designed  to  collect  data  concerning  their  mental  models  of  the 
MacFun  course.  The  subjects'  mental  model  construct  was  treated  as  a  predictor  variable  with 
respect  to  their  content  exposure  and  subsequent  performance,  and  therefore  subjects  had  to 
complete  the  PreModule  Questionnaire  before  actually  beginning  the  target  Desktop 
Operations  module.  Each  subject's  completion  of  this  instrument  was  assured  by  requiring  the 
entry  of  a  specific  password  that  only  the  researcher  knew  before  the  module  could  be  entered. 

The  teach-back  technique  used  by  Van  Der  Veer  and  his  associates  (cf.,  1989)  was  used  in 
the  PreModule  Questionnaire.  This  technique  is  an  approach  to  elicit  the  mental  models  of 
users  via  asking  users  to  explain  a  computer  program  to  new  learners.  They  should  describe  the 
program  clearly  and  with  as  much  detail  as  possible  so  that  it  is  understandable  to  new  learners. 
Three  open-ended  teach-back  questions  presented  via  a  paper-and-pencil  instrument  were  used 
to  collect  multiple  perspectives  of  subjects'  mental  models: 

1.  Give  an  overview  of  the  MacFun  course  in  such  a  way  that  it  is 
understandable  to  a  new  learner  without  experience  with  computerized 
tools  or  computer-based  learning  materials. 

2.  Suggest  seme  guidelines  for  how  to  navigate  through  the  MacFun  course  to 
new  learners  who  have  been  IBM  PC  or  compatible  computer  users  but 
have  never  used  Apple  Macintosh  computers  before? 

3.  How  would  you  explain  to  new  learners  how  to  interact  with  the  following 
sample  screen  from  the  MacFun  course? 

These  questions  were  designed  to  reveal  the  learners'  perspectives  of  the  structure  of  the  whole 
course  and  their  impressions  of  the  specific  interact!^ 'e  function^  of  commonly  used  icons.  Non- 
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directive  questions  asked  subjects  to  explain  the  structure  of  the  course  and  to  identify  the 
functions  of  the  icons  used  in  the  MacFun  course.  They  were  also  asked  to  make 
recommendations  for  new  learners  of  Macintosh  computers. 

On-line  data  tracking  routines  were  incorporated  into  the  MacFun  HyperCard  stacks  to 
collect  data  regarding  what  subjects  uid  in  the  course,  i.e.,  what  screens  they  viewed,  what 
options  they  activated,  what  responses  they  made,  and  how  long  they  spent  on  each  screen. 
This  data  was  legarded  as  a  measure  of  the  exposure  to  the  content  of  the  module  experienced 
by  each  learner.  The  content  exposure  including  screens  seen  and  icons  used  as  well  as  their 
answers  to  on-line  pop-up  questions  were  automatically  recorded  into  a  HyperCard  stack 
named  "JIH"  for  each  subject.  The  data  handling  function  provided  by  the  HyperCard  based 
courseware  allowed  the  investigator  to  track  the  data  about  subjects'  activities  in  the  course. 
On-line  pop-up  questions  (Hassett,  1990;  Koubeck,  et  al.,  1987;  Norman,  1983)  were 
incorporated  into  the  Desktop  Operations  Module  of  the  MacFun  course  to  ask  for  the 
reason(s)  why  subjects  engaged  in  certain  interactive  behaviors.  Sixteen  pop-up  questions 
asked  the  subject's  explanations  for  the  reasons  why  he/she  did  not  complete  certain  activities. 
Four  questions  asked  the  subjects'  explanations  for  the  reasons  for  unnecessary  actions  in 
reaction  to  certain  screens.  If  subjects  clicked  the  "Because..."  reason,  a  box  appeared  that 
allowed  them  to  type  in  a  unique  response.   Performance  in  the  Desktop  Operations  module 
was  measured  using  the  twelve  items  of  an  on-line  performance  test  (Self-Assessment) 
incorporated  by  Apple  Computer,  Inc.  into  the  module.  The  results  of  the  test  were 
automatically  recorded  in  each  subject's  record  by  the  data  tracking  code. 

Data  Analysis 

A  qualitative  analytical  approach  was  used  to  identify  the  specific  aspects  of  users'  mental 
models.  Because  such  a  qualitative  approach  depends  upon  the  naturally  occurring 
perspecti\  es  of  the  environment  without  any  preconceived  directional  or  specific  outcomes 
(Lincoln  &  Cuba,  1985),  the  major  concern  is  with  thorough  description  and  valid 
interpretation.  This  was  accomplished  by  data  triangulation  and  cyclic  categorizing  processes 
to  increase  the  validity  (Denzin,  1978;  Goetz  &  LeCompte,  1984).  The  reliability  of  the  study 
was  threatened  by  the  fact  that  the  investigator  was  the  sole  data  interpreter.  To  counteract 
this  threat,  the  data  related  to  subjects'  mental  models  was  anaiyzed  through  a  cyclic  process  of 
analytic  induction.  An  analytic  induction  strategy  aimed  at  identifying  categories  of  phenomena 
and  the  relationships  among  them  was  employed  in  the  following  manner: 
Step  1.  First  of  all,  the  answers  to  the  teach-back  questions  were  segmented  into  idea  units  by 
the  researcher.  Each  idea  unit  and  the  ID  number  of  the  subject  who  included  the  idea  unit  in 
his/her  answers  to  the  PreModule  Questionnaire  questions  were  written  down  on  a  1"X  3" 
piece  of  paper. 
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Step  2.  After  being  identified,  all  the  idea  units  were  grouped  and  organized  as  idea  clusters  or 
groups  by  the  researcher.  For  instance,  the  idea  unit  "mouse"  and  the  idea  unit  "a  box  with  a 
ball"  were  classified  into  the  idea  cluster  labeled  "mouse."  The  researcher's  analytical  judgment 
guided  this  organization. 

Step  3,  The  idea  clusters  were  then  grouped  into  categories.  For  example,  the  idea  clusters 
"Macintosh  or  computers  as  course  content,"  "components  of  computers,"  "definitions,  functions 
of  components,"  and  "features,  capabilities,  operation,  or  application  of  computers"  were 
categorized  into  the  group  labeled  "goal/objectives."  On  the  other  hand,  the  idea  cluster 
"Macintosh  as  a  delivering  system"  was  classified  into  "hardware/ software  features"  group.  It 
should  be  obvious  that  the  researcher's  own  mental  model  obviously  influenced  this 
categorization  process. 

Step  4.  The  classification  of  categories  and  their  associated  idea  clusters  and  idea  units  was 
conducted  several  times  in  a  cyclic  manner  to  verify  patterns  in  order  to  get  more  refined 
mental  model  constructs.  The  investigator  identified  ten  major  categories  of  the  subjects' 
mental  models:  (1)  Goal/Objectives,  (2)  Course  Structure,  (3)  Specific  Contents,  (4) 
Instructional/Learning  Strategies,  (5)  Hardware/Software  Features,  (6)  Authoring  Tool,  (7) 
Window,  (8)  Icon,  (9)  Menu,  and  (10)  Pointing  Devices  -  the  mouse.  Groups  1-4  are  related  to 
the  "instructional"  aspects  of  the  MacFun  course.  Groups  of  5-6  dealt  with  features  of  the 
computer-based  instructional  "media"  itself.  The  last  four  groups,  groups  7-10  focused  on  the 
features  of  the  windows,  icons,  menus,  and  pointing  devices  interfaces. 
Step  5.  Scores  for  subjects'  mental  models  were  computed  according  to  the  accuracy  and 
richness  of  their  answers.  Any  accurate  answer  related  to  an  associated  idea  cluster  earned  one 
point.  Answers  referring  to  the  same  idea  cluster  counted  only  as  one  point.  Incorrect  answers 
earned  nothing,  e.g.,  the  subject's  answer  of  "go  to  main  menu"  in  reference  to  the  Double  Up 
Arrow  icon  was  incorrect.  The  correct  answer  should  be  "to  the  first  menu  of  this  module". 
Another  incorrect  answer  was  the  "example  how  to  do  something"  in  reference  to  the  Pre7  icon. 
The  Pre?  icon  actually  provides  users  with  the  comparison  between  the  new  operating  system 
7.0  and  the  previous  system  6. Ox.  These  two  inaccurate  answers  earned  no  points  at  all.  After 
analysis  of  the  narrative  data  that  was  taken  from  a  subject's  answers  to  the  teach-back 
questions,  a  mental  model  "score"  was  calculated. 

A  descriptive  statistic  approach  was  used  to  calculate  the  frequencies  of  each  category  and 
idea  cluster  in  the  four  levels  of  the  WIMP  interface.  Both  quality  (highest  level)  and  quantity 
(number  of  levels  described)  aspects  of  the  mental  models  were  examined  and  calculated.  The 
subjects'  answers  to  the  on-line  pop-up  questions  were  classified  into  five  categories  according 
to  the  properties  of  those  answers:  (1)  insufficient  kiiowledge/attention  of  users,  (2)  decline  of 
memory  of  users,  (3)  miscommunication  with  course  interface,  (4)  unintentional  action,  and  (5) 
intentional  action.  These  categories  are  defined  below.  Some  subjects  had  insufficient 
knowledge  about  how  to  conduct  an  interactive  activity  such  as  executing  a  simulation,  using  a 
pull-down  menu,  or  dragging  an  object.  Others  subjects  paid  insufficient  attention  to  the 


486 

4g: 


Inlcrfaccs  &.  mental  Models 


interface  because  they  did  not  notice  they  could  do  certain  things.  The  increased  cognitive 
demand  on  the  subject's  memory  may  have  forced  the  decline  of  his/her  memory.  Several 
subjects  thought  they  had  finished  all  "More  Info"  icons  on  one  screen,  or  they  got  lost  in  the 
course.  One  subject  thought  a  whole  section  completed  with  a  check  mark  beside  the  menu 
option  indicated  it  had  been  seen  before,  but  it  was  not  completed  yet.  Subjects  who 
misunderstood  the  meanings  conveyed  in  the  interface  were  classified  as  committing 
miscommunication  errors.  Accidental  clicks  were  classified  as  unintentional  actions » 
Intentional  actions  of  subjects  happened  when  they  skipped  some  sections/options  provided  in 
the  course  because  that  they  didn't  want  to,  they  knew  the  content,  they  were  curious  to  try 
something  out,  or  they  just  wanted  to  quit. 

Users'  )  ^sufficient  knowledge  of  or  attention  to  the  course  interface  led  to  omission  of 
user-program  interactive  activities.  The  decline  of  users'  memory  resulted  in  losing  their 
original  intention  and  failing  to  complete  desired  actions.  Users'  miscommunication  with  the 
program  led  to  actions  which  were  appropriate  for  the  analysis  of  the  information  conveyed  by 
course  interface  but  inaccurate  for  the  course  itself.  Unintended  actions  occurred  when  users 
accidentally  made  any  action  such  as  clicking  the  mouse  unintentionally.  Intentional  actions 
occurred  when  users  skipped  any  section  or  activity  of  the  course  based  upon  their  own 
inclinations. 

Pearson  Product-Moment  coefficients  (r)  were  calculated  to  determine  the 
interrelationships  between  the  variables  that  constituted  individual  differences,  mental  model 
constructs,  content  exposure,  and  performance  scores  in  all  possible  paired  combinations. 
Multiple  regression  procedures  were  used  to  provide  insight  into  the  relationships  among  users' 
individual  differences,  mental  models,  content  exposure,  and  performance  scores.  Multiple 
regression  procedures  were  used  for  both  descriptive  and  inferential  purposes.  First,  multiple 
regression  was  used  to  examine  the  linear  dependence  of  users'  performance  scores  on  the 
three  main  predictor  variables,  i.e.,  individual  differences,  mental  models,  and  content 
exposure.  Second,  the  question  of  whether  the  sample  observations  can  be  generalized  to 
larger  populations  was  addressed.  For  example,  the  null  hypothesis  that  the  predictor  variables 
have  no  effect  on  learning  outcomes  was  tested  at  a  pre-specified  ,01  level  of  significance, 
Cohen  and  Cohen  (1975)  recommend  the  use  of  an  alpha  level  of  ,01  for  studies  of  this  type 
when  individuals  (as  opposed  to  classes)  are  used  as  the  unit  of  analysis. 

Discussion  and  Conclusions 

For  Question  L  Ten  categories  of  users'  mental  models  of  an  interactive  videodisc  training 
CO  Tse  using  a  hypertext  format  and  WIMP  interface  have  been  identified  according  to  their 
idea  clusters.  The  ten  categories  are:  (1)  goal/objective,  (2)  course  structure,  (3)  specific 
contents,  (4)instructional/learning  strategies,  (5)  hardware/software  features,  (6)authoring 
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tool,  (7)  window,  (8)  icons,  (9)  menu,  and  (10)  mouse  as  the  pointing  device.  Given  the  training 
function  of  the  Macintosh  Fundamentals  courseware,  it  is  not  surprising  that  four  categories  are 
related  to  the  "instructional"  aspects  of  the  course.  Two  categories  dealt  with  features  of  the 
computer-based  instructional  media  itself,  viz.,  hardware/software  features  and  authoring  tool 
features.  Given  the  unique  and  powerful  interactive  interface  of  the  Macintosh  environment,  it 
is  also  not  surprising  to  find  that  four  categories  focused  on  the  features  of  the  WIMP  interface. 
The  frequencies  with  which  these  categories  were  included  in  the  mental  models  of  the  subjects 
in  this  study  varied  considerably.  For  example,  only  seven  learners  included  the  category  qf 
course  structure  in  the  mental  model  they  expressed  whereas  all  sixty-five  learners  included  the 
icon  category  of  the  WIMP  interface  in  their  mental  model. 

No  learners  included  all  ten  categories  in  the  mental  model  they  expressed.  The  range 
went  from  two  learners  who  had  as  many  as  eight  categories  included  in  their  mental  model 
down  to  one  user  who  only  had  two  categories  expressed  in  his/her  mental  model.  The  last 
four  categories  of  the  mental  model  categorization  scheme  were  further  classified  into  task, 
semantic,  syntactic,  and  interaction  levels  for  the  idea  clusters  regarding  the  WIMP  interface. 
The  highest  total  score  for  this  classification  was  30  po^^sible  idea  clusters/levels  within  the 
mental  model.  The  range  of  inclusion  went  from  one  learner  who  scored  16  down  to  two 
learners  who  scored  4.  Twenty-three  out  of  the  65  users  formed  mental  models  with  idea 
clusters  in  the  interaction  level. 

These  results  support  previous  findings  by  Ackermann  and  Greutmann  (1990)  that 
individual  variation  among  the  mental  models  held  by  computer  users  is  great.  The  results  also 
support  Norman's  (1983)  conclusion  that  each  individuaVs  mental  model  is  incomplete  in 
nature.  However,  this  study  does  not  provide  unassailable  evidence  that  the  mental  models  of 
the  specific  courseware  used  in  this  study  held  by  the  subjects  were  captured  by  the 
measurement  methods  used  in  this  study.  As  described  below,  reliability  and  validity  studies  of 
these  measurement  techniques  remain  to  be  done.  This  study  also  does  not  address  questions 
concerning  the  mental  model  the  subjects  constructed  of  the  content  of  the  target  module,  i.e., 
desktop  operations,  that  may  have  conflicted  with  or  supported  the  mental  model  they  were 
constructing  of  the  course  itself.  It  seems  reasonable  that  there  should  be  some  overlap 
between  the  two  mental  models,  especially  with  respect  to  the  features  of  the  WIMP  interface. 

Future  research  should  address  this  issue  because  it  is  very  relevant  to  the  problem  of  the 
cognitive  demands  of  interactive  instruction.   On  the  one  hand,  a  user-friendly  interface  in  the 
courseware  should  minimize  the  amount  of  cognitive  load  dedicated  to  understanding  how  to 
use  the  courseware  and  maximize  the  amount  of  cognitive  load  that  can  be  dedicated  to  the 
content  of  the  course.  The  question  of  whether  this  actually  occurs  in  intriguing,  especially 
when  there  is  so  much  similarity  between  the  features  of  the  courseware  interface  and  the 
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content  of  the  courseware.  This  similarity  could  result  in  reinforcing  support  for  some  learners 
and  interference  for  others. 

For  Question  2,  Considerable  variation  on  the  field  dependence/field  independence  learning 
style  was  found  among  this  same  of  learners,  with  sixteen  subjects  being  classified  as  field 
dependent  and  twenty-six  subjects  classified  as  field  independent.  The  learners'  ratings  of  the 
WIMP  interface  also  varied  considerably,  although  most  subjects  expressed  positive  preferences 
for  the  WIMP  interface.  Only  twelve  subjects  (18%)  expressed  either  negative  or  uninformed 
altitudes  toward  the  WIMP  interface.  Substantial  variation  existed  among  the  subjects  with 
respect  to  experience  with  computer  systems  and  experience  with  the  Macintosh  specifically. 
The  mean  value  for  computer  experience  was  43  months  with  a  standard  deviation  of  51.  The 
mean  value  of  Macintosh  experience  was  1 1  months  with  a  S.D.  of  18. 

A  statistically  significant  (p  <  .01)  positive  relationship  was  found  between  learners' 
WIMP  ratings  and  the  richness  of  their  mental  models  of  the  courseware  and  WIMP  interface. 
These  results  indicate  that  learners  who  rated  the  WIMP  interface  positively,  based  upon  their 
preexisting  attitude  toward  or  knowledge  of  it,  also  tended  to  form  richer  mental  models  in 
terms  of  both  accuracy  and  richness.  This  can  be  interpreted  to  mean  that  the  learners'  ratings 
of  a  WIMP  interface  is  one  predictor  of  the  mental  model  constructs  of  users.  Of  course,  this 
finding  may  simply  be  a  function  of  the  positive  relationship  existing  between  knowledge  of  a 
phenomenon  and  attitudes  expressed  toward  it.  In  other  words,  those  learners  who  has 
previous  knowledge  of  and  experience  with  the  WIMP  interface  expressed  the  most  positive 
attitudes  toward  it.  In  turn,  these  same  learners  were  able  to  express  more  accurate  and  richer 
mental  models  of  the  interface  because  of  their  previous  knowledge  and  experience. 

Positive  relationships  (p<  .01)  were  detected  between  learners'  WIMP  ratings  and  their 
degree  of  Macintosh  experience  and  their  amount  of  field-independence.  In  short,  field- 
independent  users  and  experienced  Macintosh  users  tended  to  value  the  WIMP  interface  higher 
than  those  subjects  who  were  classified  as  field-dependent  and  had  little  Macintosh  experience. 
The  advertising  department  of  Apple  Computer,  Inc.  should  not  find  these  findings  to  be 
surprising,  given  that  they  have  consistently  promoted  the  Macintosh  as  the  computer  preferred 
by  independent- thinking  people.  However,  there  are  multiple  plausible  interpretations  of  these 
findings,  and  one  can  quickly  become  engaged  in  "which  came  first"  arguments.  For  example, 
one  might  argue  that  field  independent  people  are  more  likely  to  choose  the  Macintosh 
computer  with  its  WTMP  interface  over  machines  with  more  structured  interfaces  (e.g.,  MS- 
DOS  systems),  and  that  this  choice  and  their  subsequent  experience  with  the  Macintosh  leads  to 
a  positive  rating  of  the  WIMP  interface.  It  is  easy  to  see  that  other  plausible  explanations  can 
be  derived  from  the  findings.  In  any  case,  the  interrelationships  among  selected  learning  styles, 
computer  experience,  WIMP  ratings,  and  mental  model  constructs  remains  to  be  clarified. 
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For  Question  3.  No  statistically  significant  relationship  was  detected  between  learners'  FD/FI 
learning  style,  preferences  for  the  WIMP  interface,  and  previous  computer  experience  and  their 
content  exposure  at  a  .01  level  These  results  indicate  that  learners'  learning  style,  interface 
preferences,  and  previous  computer  experience  were  not  significant  predictors  of  content 
exposure  within  this  type  of  interactive,  self-paced  course* 

Only  twenty-three  (35%)  of  the  learners  completed  three  fourths  of  the  maximum  284 
possible  instructional  screens  including  227  content  screens,  35  animation  demonstrations,  9 
video  segments,  and  13  pre-7  system  comparisons.  The  reasons  why  the  subjects  did  not  go 
through  most  of  the  Desktop  Operations  module  can  be  addressed  by  reviewing  their  answers 
to  the  on-line  pop-up  questions  incorporated  into  the  module.  Their  responses  were  classified 
into  three  categories: 

(a)  intentional  action  of  skipping  some  sections/options  provided  in  the  course  because  that 
they  didn't  want  to,  they  knew  the  content,  they  were  curious  to  try  something  out,  or  they 
wanted  to  quit; 

(b)  insufficient  knowledge  about  how  to  conduct  an  interactive  activity  such  as  executing  a 
simulation,  using  a  pull-down  menu,  dragging  an  object,  or  insufficient  attention  on  the 
interface  because  they  did  not  notice  they  could  do  that; 

(c)  increased  cognitive  demand  on  their  memory,  e.g.,  they  thought  they  had  finished  sixteen 
information  icons  on  one  screen,  or  they  got  lost  in  the  course.  (One  user  thought  a  whole 
section  completed  with  a  check  mark  beside  the  menu  option  indicated  it  had  been  seen  before, 
but  it  was  not  completed  yet.) 

The  factors  involved  in  reason  (a)  above  may  be  beyond  the  influence  of  course  designers, 
but  the  factors  related  to  reasons  (b)  and  (c)  can  be  affected  by  courseware  design.  Although 
this  study  does  not  provide  sufficient  detail  to  specify  what  design  features  should  be 
modified,  it  is  clear  that  there  is  room  for  improvement  in  either  the  actual  interface  instances 
and/or  the  orientation  to  these  instances  provided  in  the  course. 

Interestingly,  the  subjects  in  this  study  did  not  use  the  nine  video  segments  to  the 
maximum  extent.  Only  twenty-five  (38%)  of  the  65  users  saw  five  or  more  video  segments. 
One  explanation  concerns  users  not  noticing  or  forgetting  to  click  on  the  video  icons  because  of 
their  relatively  rare  appearance  in  the  module.  The  use  of  separate  computer  and  video 
monitors,  and  the  possible  discorfort  involved  in  moving  one's  attention  from  one  screen  to 
another  may  be  a  factor.  Most  likely,  the  content  of  video  itself  was  a  major  reason  for  users' 
failure  to  use  the  video  segments  regularly.  Gustafson,  Reeves,  and  Laffey  (1990)  found 
that  trainees  generally  gave  poor  ratings  to  the  video  aspects  of  an  earlier  version  of  this  course. 
Fifty-two  (80%)  of  the  learners  never  used  the  "Key  Services"  options  (bookmark,  notebook, 
snapshot,  glossary,  map,  and  help)  included  in  the  module.  This  result  is  supported  by  previous 
findings  (Gustafson,  Reeves,  &  Laffey,  1990)  which  reported  minimal  use  of  these  options  by 
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trainees  using  the  first  version  of  the  MacFun  course.  This  congruence  of  these  findings 
suggests  that  the  cost-effectiveness  of  including  sophisticated  student  services  features  is 
questionable.  On  the  other  hand,  perhaps  the  course  orientation  does  not  provide  learners  with 
a  sufficient  understanding  of  the  function  and  value  of  these  options. 

For  Question  4,  There  v^ere  six  content  sections  in  the  Desktop  Operations  module  named  (1) 
Introduction,  (2)  Desktop  operations  explained,  (3)  Disk,  folders,  and  files,  (4)  Using 
Macintosh  menus,  (5)  Customizing  your  system,  and  (6)  Putting  it  all  together.  (The  "Self 
Assessment"  test  was  a  seventh  option  on  the  main  screen  of  the  module.)  Subjects  were 
free  to  complete  as  much  or  as  little  of  these  sections  as  they  wished.  There  was  considerable 
variance  with  respect  to  learner  completion  of  the  various  subsections  of  the  module  ranging 
from  the  forty-seven  (72%)  of  them  who  completed  at  least  80%  of  the  available  screens  in  the 
"Putting  it  all  together"  subsection  down  to  the  twenty  (31%)  who  completed  at  least  80%  of  the 
"Introduction"  subsection. 

Perhaps  the  most  interesting  findings  related  to  the  question  of  the  relationship  between 
mental  models  and  module  navigation  involves  recent  changes  in  the  Macintosh  operating 
system.  In  the  Spring  of  1991,  Apple  Computer,  Inc.  introduced  a  major  new  operating  system 
called  "System  7."  Because  of  considerable  differences  between  this  new  operating  system  and 
the  previous  "6.0x"  operating  systems,  not  all  Macintosh  software  operates  in  the  new 
environment.  In  addition,  the  new  system  demands  much  more  RAM  memory  than  older 
system  versions.  Therefore,  many  users  have  chosen  to  continue  to  use  the  older  operating 
systems. 

The  designers  of  the  "Desktop  Operations"  module  were  faced  with  the  dilemma  of 
deciding  how  to  handle  content  differences  between  the  old  and  new  operating  systems.  They 
opted  to  put  System  7  procedures  into  the  main  sections  of  the  module,  and  to  provide  an  "pre7" 
option  in  the  form  of  an  icon  on  the  navigation  sidebar  whenever  it  was  relevant.  A  positive 
relationship  (p<  .01)  was  found  between  the  levels  of  users'  mental  models  and  the  number  of 
pre-7  options  used  by  them.  This  may  be  explained  by  the  positive  relationship  existing 
between  previous  Macintosh  experience  and  mental  models  described  under  question  2  above. 
Those  learners  with  previous  experience  with  the  Macintosh  most  likely  gained  that  experience 
using  a  "Pre7"  version  of  the  operating  system  since  System  7  had  been  available 
only  a  short  time  when  this  study  was  conducted.  Hence,  they  were  the  subjects  most  likely  to 
want  to  compare  the  new  system  with  the  one  they  already  knew.  Less  experienced  users  of  the 
Macintosh  (who  also  possessed  less  complete  mental  models  of  courseware  and  WIMP 
interface)  were  less  likely  to  want  to  explore  these  differences. 

In  summary,  it  seems  reasonable  to  conclude  that  learners  with  a  richer  mental  model  of 
the  course  will  explore  more  of  the  course  content  than  those  with  less  complete  understanding 


491  4G3 


Znterfaces  &  mental  Modeig 


14 


of  the  course  structure  and  interactive  options.  An  alternative  explanation  may  be  that 
experienced  Macintosh  users  are  more  likely  to  have  richer  mental  models  and  also  have  a 
greater  desire  to  explore  the  topic  of  "Desktop  Operations."  Additional  research  is  indicated 
with  radically  different  content,  preferably  content  that  has  little  or  nothing  to  do  with  the 
actual  courseware  delivery  system. 

For  Questions  5.  As  described  above^  learners  who  had  more  experience  with  the  Macintosh 
tended  to  rate  the  WIMP  interface  higher,  learners  whose  WIMP  ratings  were  higher  tended  to 
form  richer  levels  of  n<ental  models,  users  with  a  richer  mental  models  construct  tended  to  view 
more  comparisons  between  new  "System  7"  operation  procedures  and  those  employed  in 
previous  system  6.0x  versions.  Not  surprisingly,  these  same  learners  tended  to  earn  higher 
performance  scores.  However,  the  multiple  regression  results  derived  from  the  examination  of 
the  predictive  value  of  individual  differences,  mental  models,  and  content  exposure  on 
performance  were  not  impressive.  WIMP  ratings  and  the  number  of  pre-7  options  used  were 
the  only  two  significant  predictors  of  performance  score  {p<  .01). 

It  was  still  uncertain  what  kind  of  interrelationship  exists  among  these  variables  or  what 
other  factors  might  be  involved  in  predicting  performance  in  this  type  of  hypertext  courseware. 
There  are  several  flaws  in  the  research  design  that  may  have  prohibited  stronger  findings. 
Although  the  mean  time  of  over  three  hours  actually  spent  in  the  MacFun  course  far  exceeds 
the  30-60  minutes  of  treatment  used  in  most  research  of  this  kind  (Clark,  1989),  it  may  have 
been  insufficient  for  some  of  the  learners  to  develop  a  sufficient  mental  model  of  the  course. 
Another  problem  stems  from  the  fact  that  the  subjects  in  this  study  were  volunteers.  It  is  much 
more  desirable  to  conduct  this  type  of  study  with  trainees  or  students  who  have  relatively  strong 
intrinsic  or  extrinsic  motivations  for  participation  in  a  course.   Sample  size  is  also  a  weakness  in 
this  study.  Further  research  employing  multiple  regression  techniques  with  larger  sample  sizes 
might  yield  richer  data  to  address  the  major  research  questions. 

Another  problem  with  this  study  relates  to  the  validity  of  the  performance  measure,  i.e., 
the  "Self  Assessment"  test  built  into  the  "Desktop  Operations"  module.  Perhaps  users  who  saw 
more  screens  and  accessed  more  animation  demonstrations  and  video  segments  did  "learn 
more"  from  the  content  of  the  screens,  videos,  and  animations,  but  failed  to  demonstrate  the 
performance  on  the  Self-Assessment  test.  The  questions  in  this  test  covered  only  three  of  six 
subsections  of  the  module  and  did  not  reflect  the  video  content  in  the  Desktop  Operations 
module.   A  more  comprehensive  test  with  stronger  articulation  between  the  module  content 
and  the  test  items  might  yield  more  impressive  findings.  The  weak  findings  of  the  predictive 
power  of  mental  models  with  respect  to  performance  did  not  support  the  findings  of  previous 
studies  on  facilitating  learning  performance  (e.g.,  Brehmer,  1987;  Mayer,  1989;  Moray,  19S7). 
One  possible  explanation  is  that  users*  mental  models  may  be  more  predictive  in  more  complex 
computer  environments,  such  as  a  simulation  program,  a  problem  solving  program,  a  data  base 
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application,  or  a  design  tool  which  requires  users  to  apply  more  sophisticated  interactive 
skills/techniques.  Still,  considerable  variation  was  found  in  the  mental  models  held  by  the 
subjects  in  this  study,  and  it  would  be  interesting  to  examine  the  relationship  between  these 
mental  models  and  a  better  performance  measure. 

Implications  of  This  Study 

Based  upon  the  results  and  conclusions  of  this  study,  several  implications  are  evident: 

1.  The  coded  inspection  capabilities  for  users'  content  exposure  and  the  on-line  pop-up 
questions  used  in  this  study  could  serve  as  a  new  feature  of  program-control  to  advise 
learners  to  go  through  certain  "essential"  content  and/or  orientation  activities  within 
computer-based  learning  systems  that  permit  the  degree  of  learner  control  found  in  the 
Macintosh  Fundamentals  course. 

2.  The  constructs  of  the  mental  models  identified  in  this  study  could  serve  as  a  tool  for  the 
purpose  of  courseware,  manual,  and  tutorial  evaluations  as  well  as  a  basis  for  further 
research  on  learner-courseware  interactions  in  computer-based  learning  systems. 

3.  The  teach-back  technique  and  the  on-line  pop-up  questions  could  serve  as  a  new 
methodology  for  examining  learner-courseware  interactions  in  computer-based  learning 
systems. 

4.  The  need  to  add  many  student  services  functions  such  as  notetaking,  snapshot,  and 
bookmarking  in  a  computer-based  learning  system  should  be  examined  with  respect  to  the 
cost-effectiveness  of  their  inclusion. 

5.  Video  segments  in  the  these  types  of  hypermedia  interactive  courses  should  be  integral  to  the 
content  and  activities  of  the  course*  If  they  are  not  integral  to  the  courx,  learners  will  most 
likely  ignore  them.  The  higher  expense  involved  in  producing  and  providing  video  in  this 
type  of  courseware  makes  it  all  the  more  important  that  these  components  are  essential. 

6.  A  better  course  orientation  should  be  designed  including  more  descriptions  and  practice 
with  all  of  the  interactive  techniques  such  as  how  to  use  mouse  to  drag  an  object,  how  to  use 
pull-down  menus,  etc.  More  importantly,  the  orientation  should  demonstrate  the  use  and 
value  of  the  student  services  functions,  and  the  utility  of  animation  and  video  segments. 
Courseware  designers  should  let  users  practice  the  skills  of  navigating  through  the  course 
and  support  their  navigation  with  a  users'  manual,  on-line  tutorial,  and/or  on-line  help 
information. 

7.  Descriptions  and  demonstrations  about  the  windows,  icons,  menus,  and  pointing  devices 
interfaces  should  be  one  of  the  major  options  in  courseware  driven  by  a  WIMP  interface  to 
enhance  users*  attitude  toward  and  knowledge  on  the  specific  interface  as  well  as  their 
performance  scores. 

8.  In  any  screen  with  many  icons/options  available  in  this  type  of  course  or  other  program, 
memory  aids  should  be  provided  to  indicate  user  status.  For  example,  a  check  mark,  a 
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dimmed  icon,  or  a  semi-darken  icon  can  be  employed  to  make  up  for  users*  decline  of 
memory, 

9.  In  the  menu  options  screen,  the  combination  of  advisement  for  users'  progress  such  as  a 

sentence  of  "You  completed  %  of  this  section"  or  a  progress  bar  and  the  check  mark  " 

could  help  decrease  the  users'  cognitive  load  (Peters,  1989). 

Recommendations  for  Further  Studies 

The  following  recommendations  and  suggestions  for  further  research  are  based  on  the 
results  of  this  study: 

1 .  A  study  replicating  this  effort  with  actual  trainees  and  larger  sample  sizes  should  be 
conducted. 

2.  A  study  using  a  single-screen  delivery  platform  with  a  WIMP  driven  interface  for  instance, 
an  IBM  PS/2t  driven  by  Windows  or  a  Macintosh  II  with  a  video  card  for  single  screen 
integration,  should  be  undertaken  to  ascertain  whether  results  are  similar  to  those  found  in 
this  study. 

3.  This  study  looked  at  users'  field  independence,  their  experiences  with  computers,  and  their 
WIMP  interface  ratings  as  variables  of  individual  differences.  Other  variables  shoulf'  be 
examined  such  as  learning  needs  or  goals,  computer  experience  other  than  the  instrument 
type,  and  users'  self-esteem  with  respect  to  their  computer  literacy. 

4.  More  data  resources  of  users'  mental  models  and  interactive  behaviors  collected  by  a  variety 
of  approaches  should  be  explored,  e.g.,  observing  users  learning  to  navigate  through  a 
courseware  under  the  guidance  of  manuals  or  a  front-end  tutorial  program,  asking  users  to 
explain  the  courseware  to  a  new  learner  as  the  co-experimenter,  and/or  observing  the  users 
and  co-experimenter  navigating  through  other  sections  of  the  courseware  or  another  similar 
course  via  verbal  protocol,  questionnaires,  interviews,  and  on-line  computer  coding.  Such 
research  could  yield  a  taxonomy  of  users'  mental  models  in  a  hypermedia  learning 
environment,  the  patterns  of  their  content  exposure,  and  other  factors  possibly  existing  in  the 
human -computer  interactions. 

5.  A  study  should  be  conducted  to  determine  idea  clusters,  categories,  and  levels  of  mental 
models  held  by  designers  of  interactive  courseware.  These  results  should  be  compared  with 
the  results  of  the  mental  models  formed  by  users/learners  using  the  course  these  designers 
created. 

6.  This  study  should  be  repeated  with  the  same  interface  but  radically  different  content. 
Assumptions  and  Limitations 

Assumptions 

1.  Previous  experience  with  computers  is  a  reasonable  estimate  of  a  subject's  expertise. 
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2.  The  personal  constructs  or  mental  models  that  learners  construct  of  the  course  or  WIMP 
interface  can  be  assessed  via  teach-back,  paper-and-pencil  questions.  User's  answers  to  teach 
back  questions  can  accurately  reflect  their  mental  models. 

3.  The  subjects  who  ranged  in  education  attainment  from  a  high  school  diploma  to  a  Ph.D. 
degree  had  the  capabilities  to  describe  the  MacFun  course  in  such  a  way  that  it  is 
understandable  to  other  new  learners. 

4.  The  subjects  gave  the  real  explanations  for  the  reasons  why  they  did  not  complete  certam 
activities  or  unnecessary  actions  in  reaction  to  certain  screens. 

Limitations 

1.  One  general,  fundamental  limitation  is  that  the  conceptualization  of  learners'  mental  models 
is  dependent  on  the  investigator's  and/or  the  courseware  designers*  own  mental  models. 

2.  The  MacFun  interactive  courseware  consists  of  thirteen  modules.  The  subjects  of  this  study 
were  able  to  control  their  own  pace  and  learning  paths  through  the  sections  of  the  module, 
therefore  the  navigational  history  and  maturation  of  the  subjects  in  the  particular  courseware 
continually  changed.  Mental  models  are  more  likely  to  be  dynamic  entities  and  have  multiple 
forms.  The  collected  users'  understanding  about  the  courseware  in  terms  of  a  mental  model  is 
only  one  of  the  multiple  models  that  can  exist  over  time  for  the  specific  individual.  The  f  igs 
may  not  generalize  to  either  smaller,  simpler,  larger,  or  more  complex  courseware  in  terms  of 
the  size  and  structure  or  to  less  learner-controlled  programs. 

3.  There  was  a  possible  contamination  of  the  treatment  in  that  some  subjects  may  have  been 
exposed  to  the  design  of  the  MacFun  course  before  participating  in  this  study.  This  previous 
exposure  could  have  occurred  in  the  context  of  a  graduate  course  such  as  EIT  714  -  Design  and 
Development  of  Interactive  Instructional  Materials.  Or  they  may  have  viewed  parts  of  the 
courseware  in  a  laboratory  setting  open  to  many  of  the  students  used  as  subjects  in  this  study. 

4.  The  amount  of  time  that  subjects  spent  in  the  MacFun  course  before  eliciting  their  mental 
model  varied  considerably  (mean  time  =  82  minutes,  standard  deviation  =  44  minutes).  The 
briefest  amount  of  time  was  1  minute  and  the  longest  amount  of  time  was  220  minutes. 
Whether  subjects  who  spent  relatively  brief  periods  in  the  course  before  their  mental  models 
were  assessed  had  sufficient  time  to  construct  a  meaningful  mental  model  is  not  known. 

5.  The  user-courseware  interface  investigated  in  this  study  consists  of  WIMP  patterns  on  a 
mor  jchrome  monitor  of  the  Macintosh  computer  system.  The  findings  may  not  generalize  to 
Macintosh  color  monitors  or  to  other  computers  using  WIMP  interfaces. 

6.  The  delivery  system  for  the  MacFun  course  includes  one  Macintosh  computer  with  its  own 
monitor  and  a  separate  monitor  for  display  of  videodisc  material.  The  findings  may  not 
generalize  to  single  screen  computer  systems. 
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Effects  of  Building  Expert  System  Rule  Bases 
on  Cognitive  Structures 
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Abstract 

Researchers  and  businesses  are  investigating  the  systems  and  cognitive  effects  of 
having  students  and  employees  construct  expert  system  rule  bases  that  represent  their 
knowledge.  Expert  systems  are  cognitive  tools  that  engage  learners  in  cognitive  and 
metacognitive  learning  outcomes.  This  study  investigated  the  changes  in  domain-specific 
cognitive  structure  that  result  from  students  constructing  rule  bases.  These  changes  were 
idiosyncratic  and  not  related  to  the  expert's  (teacher's)  cognitive  structure. 

Expert  Systems  as  Mindtools 
Rule-based  exf>ert  systems  are  an  application  of  artificial  intelligence,  where  the 
knowledge  of  an  expert  is  distilled  into  a  set  of  rules  that  can  be  used  to  guide  decision  making 
in  solving  problems.  Expert  systems  are  useful  tools  that  are  used  extensively  in  business  to 
help  users  make  decisions.    However,  they  may  support  learning  and  instruction  in  a  variety 
of  ways  (Jonassen,  Wilson,  Wang  &  Grabinger,  in  press),  including  expert  and  student 
modelling  for  intelligent  tutors,  instructional  design  advisors,  internally  generated  feedback, 
cognitive  simulations,  and  cognitive  amplification  tools  or  mindtools  (see  Figure  1).  The 
research  reported  in  this  paper  focuses  on  the  mindtool  application  of  expert  systems. 
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Range  of  Expert  System  Applications 
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Mindtools  are  computer-based  application  tools  that  engage  learners  in  critical 
thinking.  They  include  expert  systems,  databases,  constructive  hypertext,  semantic 
networking,  spread  sheets,  and  microworlds  of  various  types.  When  used  to  analyze  content 
structures  of  learning  materials  or  personal  knowledge  structures,  mindtools  function  as 
computer-based  learning  strategies  that  engages  and  supports  generalizable  cognitive 
processing  (Jonassen,  in  press;  Kommers,  Jonassen  &  Mayes,  1992).  Mindtools  can  be  used  to 
analyze  or  represent  a  subject  matter,  or  to  help  learners  organize  their  thoughts  as  they 
prepare  to  write  a  paper  to  produce  a  product.  Such  tools  help  to  make  abstract  subjects  and 
ideas  more  concrete  and  manipuiable. 

Cognitive  Outcomes  of  Expert  System  Construction 

Expert  systems  can  serve  as  computer-based  mindtools  that  a  learner  can  flexibly  make 
use  of,  similar  to  hypertext,  semantic  mapping,  and  other  organizational  tools.  The  analysis 
of  subject  matter  required  to  develop  expert  systems  is  so  deep  and  so  incisive  that  learners 
develop  a  greater  comprehension  of  their  subject  matter,  engaging  in  analytical  reasoning, 
elaboration  strategies  such  as  synthesis,  and  metacognitive  strategies  such  as  validating 
(Jonassen  &  Grabinger,  1992).  Building  expert  system  knowledge  bases  is  a  generative  and 
constructivistic  process  that  facilitates  the  acquisition  of  conceptual  as  well  as  procedural 
knowledge.  TroUip,  Lippert,  Starfield,  &  Smith  (1992)  believe  that  the  development  of  expert 
systems  results  in  deeper  understanding  because  they  provide  an  intellectual  environment 
that: 

•  demands  the  refinement  of  domain  knowledge, 

•  supports  problem  solving,  and 

•  monitors  the  acquisition  of  knowledge. 

The  first  requirement,  the  refinement  of  domain  knowledge,  is  met  because  building 
expert  systems  requires  generative  processing  of  information.  According  to  the  generative 
hypothesis  (Wittrock,  1974),  information  becomes  meaningful  to  the  individual  insofar  as  it  is 
related  to  prior  knowledge.  Building  expert  systems  requires  the  knowledge  engineer  to 
expose  the  prior  knowledge  of  the  expert.  This  entails  identifying  declarative  knowledge  (facts 
and  concepts),  structural  knowledge,  the  knowledge  of  the  interrelationships  of  ideas  in 
memory  (Jonassen,  Beissner,  &  Yacci,  in  press),  and  procedural  knowledge  (how  to  apply  the 
former).  Building  experts  systems,  we  believe,  entails  converting  existing  declarative 
knowledge  into  procedural  knowledge  and  applying  it.  Finally,  Trollip  et  al  believe  that 
environments  should  provide  a  mechanism  for  monitoring  their  own  knowledge.  This 
entails  metacognitive  awareness  of  their  knowledge,  which  is  the  highest  order  of  intellectual 
processing  (Flavell  et  al,  1977). 

An  important  distinction  of  building  expert  systems,  vis-a-vis  other  mindtools,  is  that 
the  process  concentrates  on  depicting  procedural  knowledge,  whereas  others  focus  on 
declarative  and  structural  knowledge  changes.  Psychologists  usually  represent  procedural 
knowledge  as  a  series  of  IF-THEN  rules  (E.  Gagne,  1985);  such  a  representation  mode  is 
obviously  well-suited  to  expert  system  codification.  As  learners  identify  the  IF-THEN 
structure  of  a  domain,  they  will  tend  to  understand  the  nature  of  decision  making  tasks  better; 
this  deeper  imderstanding  should  make  subsequent  practice  opportunities  more  meaningful. 
We  would  not  want  to  suggest  that  the  mere  development  of  an  expert  system  necessarily 
leads  learners  to  acquire  the  compiled  procedural  knowledge  of  a  domain.  Merely  identifying 
the  IF*THEN  rules  involved  in  a  process  does  not  ensure  acquisition  of  the  procedural 


expertise,  which  would  require  extended  practice  opportunities  in  realistic  performance 
settings. 

Finally,  Trollip  et  al.  (1992)  believe  that  environments  should  provide  a  mechanism  for 
monitoring  their  own  knowledge.  Reflecting  on  what  they  know  using  rule-based  expert 
systems  as  a  formalism  requires  learners  to  reconceptualize  and  restructure  their  knowledge 
as  well  as  simply  making  learners  aware  of  how  much  they  do  know.  This  process  entails 
metacogiutive  awareness  of  their  knowledge,  v/hich  is  a  necessary  component  of  problem 
solving  (Flavell,  1976), 

Assessing  Knowledge  Structures 

Structural  knowledge  is  the  knowledge  of  how  concepts  within  a  domain  are 
interrelated  (Diekhoff,  1983).  If  schemas  are  defined  by  their  interrelationships  to  other 
schemas,  then  explicit  awareness  of  those  interrelationships  and  the  ability  to  explicate  those 
relationships  is  essential  for  higher  order,  procedural  knowledge.  It  is  not  enough  to  know 
that.  In  order  to  know  how,  you  must  know  why.  Structural  knowledge  provides  the 
conceptual  bases  for  why;  it  describes  how  the  declarative  knowledge  is  interconnected. 
Procedural  knowledge  is  predicated  on  these  interconnections  between  ideas.  Structural 
knowledge  enables  learners  to  form  the  connections  that  they  need  to  describe  and  use  scripts 
or  complex  schemas. 

Structural  knowledge  has  also  been  described  as  "cognitive  structure",  the  pattern  of 
relatio.iships  among  concepts  in  memory  (Preece,  1976).  Cognitive  structure  in  learners  is  a 
reflection  of  subject  matter  structure.  Structural  knowledge  can  be  thought  of  as  the 
understanding  of  one's  cognitive  structure. 

This  research  seeks  to  identify  the  structural  knowledge  outcomes  of  building  expert 
systems,  that  is,  the  degree  of  individual  knowledge  restructuring  that  occurs  as  a  result  of 
building  expert  systems.  To  what  extent  is  knowledge  restructured  when  learners  synthesize 
procedural  understanding  by  making  explicit  their  own  knowledge  structures  or  the 
knowledge  structures  of  experts. 

METHOD 

Assessment  Instruments 

The  cognitive  structures  of  seventeen  students  in  a  graduate  education  course  on 
individual  differences,  learning  and  instruction  were  pretested  using  relationship  proximity 
judgments.  Perhaps  the  most  accurate  method  for  assessing  structural  knowledge  is  the  use  of 
Pathfinder  networks  (Goldsmith,  Johnson,  &  Acton,  1991).  Pathfinder  Nets  use  a  scaling 
algorithm  for  transforming  semantic  proximity  matrices  into  network  structures  in  which 
each  object  in  the  structure  is  represented  by  a  node  in  the  network  and  the  relatedness 
between  objects  is  depicted  by  their  physicd  closeness  (Schvanenveldt,  Durso,  &  Dearholt, 
1985;  1987;  1989).  The  algorithm  searclies  through  the  nodes  to  find  the  closest  indirect  path 
between  objects.  The  algorithm  retains  only  the  links  with  a  minimum-length  path  between 
two  concepts,  so  in  the  resvilting  net,  not  all  concepts  are  not  directly  connected  to  all  other 
concepts,  as  is  implied  by  other  types  of  dimensional  representations. 

FroceJtuce 

The  research  was  conducted  in  the  context  of  a  course  on  analyzing  learner 
characteristics  and  relating  them  to  learning  outcomes  and  instruction  techniques.  On  the 
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first  night  of  dass,  students  assessed  their  knowledge  structures  related  to  content  domain. 
Pathfinder  networks  were  generated  by  each  of  the  subjects  before  instruction.  This  involved 
each  of  the  students  rating  the  semantic  relatedness  of  23  concepts  that  were  most  integral  to 
the  instructional  content  (see  Table  1).  The  rating  program  asks  the  student  to  rate  the 
similarity  of  each  pair  of  concepts  on  a  scale  of  1-9,  with  1  representing  low  or  no  relatedness 
and  9  representing  a  high  level  of  relatedness.  This  process  v/as  difficult  because  some  of  the 
concepts  were  new  to  the  student. 


learning 
instruction 
individual  differences 
intelligence 
cognitive  styles 
cognitive  controls 
personality 
prior  knowledge 


learning  styles 
mental  abilities 
field  independence 
cognitive  flexibility 
impulsivity/reflectivity 
cognitive  complexity 
category  width 
visual/haptic 


leveling/sharpening 
anxiety 

frustration  tolerance 
locus  of  control 
achievement 
risk  taking 
structural  knowledge 


Table  1 

Key  Concepts  in  Pathfinder  Networks 


After  pretesting,  students  in  the  class  were  taught  to  use  a  production  rule  expert 
system  shell,  Primex  1,  which  consisted  of  an  editor,  Prolog  compiler,  and  a  set  of  advice 
windows  which  provided  the  advice,  explanations  of  the  reasoning,  and  traces  of  the  sessions. 
The  only  assignment  and  requirement  for  the  course  was  to  develop  a  medium  sized  expert 
rule  base  (200-300  rules)  which  would  provide  a  novice  teacher  advice  on  which  instructional 
strategies  would  be  appropriate  to  use  to  teach  students  with  different  learner  characteristics  a 
set  of  4r5  stated  learning  outcomes.  This  task  required  evaluating  each  learner  characteristics 
for  possible  interactions  with  stated  learning  outcomes,  identifying  possible  instructional 
strategies  that  could  be  used  to  capitalize,  challenge  or  remediate  those  different  learners.  At 
the  conclusion  of  tlie  course,  they  were  posttested  with  both  instruments. 

In  each  of  the  subsequent  eight  classes,  learners  self-evaluated  themselves  on  mental 
abilities,  cognitive  control,  cognitive  style,  learning  style,  and  personality  instruments  and 
then  discussed  their  conceptions  of  eadi  of  the  individual  differences  listed  below: 


fluid  intelligence 
structure  of  intellect  abilities 
field  independence 
cognitive  flexibility 
impulsivity/reflectivity 
category  width 
visualizer/verbalizer 


leveling  /  sharpening 
conceptual  style 

Grasha-Reichman  learning  styles 
Gregorc  learning  styles 
anxiety 

ambiguity  tolerance 
locus  of  control 


•  extraversion/introversion 

Discussions  also  focused  on  the  types  of  learning  outcomes  that  would  most  likely  interact 
with  each  of  the  individual  differences  and  the  types  of  teaching  methods  that  could  be  used 
to  preferentially  and  compensatorally  match  instriiction  to  each  individual  difference.  That 
is,  the  discussions  supported  the  reasoning  necessary  for  building  their  knowledge  bases.  The 
students  were  building  their  knowledge  base  as  the  instruction  and  discussions  occurred.  By 
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the  end  of  instruction,  each  student  had  completed  at  least  200  rules  and  most  had  completed 
substantially  more. 

At  the  end  of  instruction,  students  generated  proximity  ratings  and  Pathfinder  nets  to 
assess  how  their  knowledge  structures  had  changed  as  a  result  of  the  course. 

RESULTS 

Pathfinder  Nets  generated  by  each  student  at  the  end  of  the  course  were  compared  with 
the  students'  nets  at  the  beginning  of  the  course.  Both  pretest  and  posttest  nets  were 
compared  with  an  experts  representations  of  the  content  using  the  same  constrained  set  of 
concepts  (see  Figure  1  for  the  layout  of  the  Pathfinder  net).  Finally,  each  student's  beginning 
and  ending  nets  were  compared  with  a  net  that  was  generated  to  represent  the  causal  strength 
of  relations  between  each  of  the  concepts*  That  is,  similarity  ratings  represented  the  degree  of 
causality  implied  by  the  relationship  between  concepts  (see  Figure  2  for  a  layout  of  the 
Pathfinder  net). 
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Figure  1 
Expert  Pathfinder  Net 
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These  comparisons  were  produced  using  a  set-theoretic  m<2thod  for  quantifying  the 
configural  similarity  between  two  nets  using  a  common  set  of  nodes  known  as  C.  The  C 
measure  exanaines  the  degree  to  which  the  same  node  in  two  graphs  with  a  common  set  of 
nodes  is  surrotmded  by  a  similar  neighborhood  of  nodes. 
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igure  2 

Pathfinder  Net  Representing  Causal  Links 

This  neighborhood  comparison  is  performed  for  each  node  in  the  two  graphs  and  the  results 
averaged  across  nodes  to  compute  an  overall  similarity  index.  Two  graphs  are  similar  to  the 
degree  that  the  patterns  of  edges  that  define  how  two  graphs  are  linked  are  the  same.  The 
edges  describe  the  paths  that  exist  between  the  nodes.  The  C  sinularity  rating  measure  has 
been  found  to  be  more  predictive  of  final  course  grades  than  raw  proximity  ratings  and 
multidimensional  scaling  solutions  (the  most  accurate  method  used  previously),  and  as  a 
measure  of  structiu-al  similarity,  it  is  consistent  with  a  configural  notion  of  cognitive  structure 
(Goldsmith,  Johnson,  &  Action,  1991). 

Pretest-posttest  comparisons  and  expert-posttest  comparisons  were  averaged  across  each 
group  and  the  groups  means  were  compared  using  t  tests,  Similarities  were  calculated  for 
student's  beginning  and  ending  Pathfinder  nets.  ITie  nets  showed  some  similarity 
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(M=.238(.115)),  indicating  that  while  students'  conceptions  of  the  knowledge  domain  dearly 
changed,  the  posttest  were  still  affected  by  the  pretest  ratings.  Similarity  between  students' 
pretest  and  postest  nets  and  the  expert  net  were  calculated.  The  average  similarity  rating 
between  students'  nets  and  the  expert  nets  increased  significantly  from  pretest  (Mr=.149)  to 
posttest  (Mr=.187),  t=2.485,  p<.05.  Next,  similarity  between  students'  pretest  and  postest  nets 
and  the  causal  model  net  were  calculated,  but  the  average  similarity  rating  s  did  not  increase 
significantly  from  pretest  to  posttest,  t=.917,  p>.05. 
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Figures 
Student  Pretest  Pathfinder  Net 
Additionally,  the  nets  were  compared  visually  in  order  to  interpret  the  reasoning  of  the 
students.  For  instance.  Figure  3  shows  one  of  the  student's  pretest  nets  in  which  the  concepts 
are  clustered  together  in  an  undistinguished  manner.  The  clusters  do  not  desaibe 
meaningful  or  appropriate  relationships  between  the  concepts.  The  connections  between  the 
nodes  are  confused  and  inappropriate.  There  is  no  central  node.  Posttest  nets,  on  the  other 
hand,  contained  multiple  clusters  with  logical  connections  between  the  nodes.  In  this  case, 
achievement,  learning  and  instruction  are  the  central  nodes,  indicating  more  similarity  to  the 
casual  model  net  in  Figure  2. 
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Figure 


Student  Posttest  Pathfinder  Net 


DISCUSSION 


It  is  clear  that  the  cognitive  structures  of  the  learners  changed  as  a  result  of  instruction. 
They  changed  significantly  toward  the  teacher/expert's  model  cognitive  structure,  which  is 
similar  to  other  research  which  showed  that  students'  cognitive  structures  become  more 
congruent  with  cin  expert's  structure  as  a  result  of  explicit  instruction  (Shavelson,  1972,  1974; 
Shavelson,  RJ-  &  Stanton,  1972;  Shavelson,  RJ.  &  Geeslin,  1972).  However,  this  study  sought 
to  determine  how  student's  cognitive  structxires  would  be  influenced  by  the  formalism  that 
they  were  using  to  represent  their  knowledge,  in  this  case,  expert  systems.  Relative  to  that 
question,  no  consistent  effects  could  be  determined.  That  s,  strength  of  causal  relations  were 
not  eviden  tin  the  nets  that  students  generated  at  the  end  of  instruction.  While  the  students 
knowledge  structures  dearly  changed,  those  changes  were  mediated  by  a  variety  of  individual 
differences,  which  ironically  was  the  topic  of  the  course  in  which  the  were  working. 

Obviously,  this  study  needs  to  be  replicated  using  other  formalisms  to  ascertain 
whether  or  not  different  representation  formalisms  produce  different  structural  changes  n 


4:0 


506 


students  memory.  That  study  is  planned,  using  a  semantic  networking  fomialism  (using  the 
SemNet  program).  Comparing  the  effects  of  different  representation  formalisms  on  cognitive 
structural  changes  will  hopefully  provide  evidence  about  the  strength  of  these  formalisms. 
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Abstract 

A  tool-supported  methodology  to  assist 
in  the  creation  of  complete  instructional 
systems  is  introduced.  The  three 
disciplines  of  software  engineering, 
artificial  intelligence,  and  instructional 
design  are  integrated  to  create  an 
Intelligent  Authoring/Instructional 
System  (lA/IS).  Then,  the  ten-step 
methodology  is  specified.  Acquisition 
tools  provided  in  the  lA/IS  are  used  by 
the  authoring  team  to  extract  the 
knowledge  required  in  the  lA/IS's  three 
co-operating  knowledge-bases.  This 
system  is  an  intelligent  courseware 
development  environment  as  well  as  an 
intelligent  tutoring  system. 


1.  Introduction 

There  is  very  little  in  the  literature  about 
a  methodology  for  creating  an  ITS.  Two 
articles  were  found  that  describe  a 
methodology  related  to  ITS.  Clancy's 
article  [CLAN87]  describes  the 
methodology  used  to  develop 
NEOMYCIN  and  could  be  generalized  to 
apply  to  developing  any  diagnostic 
expert  system,  but  it  can  not  be 
generalized  to  apply  to  tutoring  systems 
in  general.  Baker  [BAKE91]  discusses  the 
task  of  developing  a  methodology  for 
assessing  the  technology  used  in  ITS. 


She  states  that  one  of  the  reasons  there  is 
so  little  empirical  validation  of  the 
existing  ITS  is  that  most  of  the  research 
has  been  so  focused  on  one  specific 
module  of  the  system  that  there  are  very 
few  whole  systems  that  have  been 
produced.  When  it  becomes  feasible  for 
courseware  designers  to  create 
instructional  systems  in  a  timely  and 
cost-effective  manner,  the  increase  in  the 
number  of  complete  instructional 
systems  created  will  make  it  possible  to 
gather  the  empirical  evidence  needed  to 
properly  evaluate  the  instructional 
principles  incorporated  into  these 
systems.  To  address  some  of  the 
problems  associated  with  the  creation  of 
these  complex  systems,  this  paper 
introduces  an  integrated  tool-supported 
methodology  for  authoring  ITS. 

In  Section  2  of  this  paper,  i  discuss  the 
rationale  for  using  an  integrated  tool 
supported  methodology  and,  in  Section 
3,  the  system  architecture  for  my  model 
is  introduced.  The  ten  step  methodology 
is  presented  in  Section  4.  Then,  the 
system  is  evaluated,  first,  as  an 
intelligent  instructional  system  in 
Section  5,  and  then,  c.s  an  authori.i^ 
system  in  Section  6.  A  prototype  of  this 
system  is  described  in  Section  7,  and, 
finally,  my  conclusions  are  presented  in 
Section  8. 
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2.  Integrated  Methodology 
In  order  to  build  an  ITS  it  is  necessary  to 
integrate  three  different  disciplines; 
software     engineering,  artificial 
intelligence,  and  education. 

•Software  engineering  is  needed  to 
develop  a  methodology  and  a  set 
of  tools  to  provide  assistance  in 
building  these  complex  systems  in 
a  timely,  cost-effective  way  The 
educational  effectiveness  of  any 
tutoring  system  is  enhanced  by  the 
inclusion  of  appropriate  graphics, 
animations,  and  sound  that  can  be 
created  using  authoring  tools. 

•Artificial  Intelligence  techniques 
are  necessary  in  order  to  create  ^n 
adaptive  system  that  models  the 
student  and  monitor  his  progress. 

•Educational  considerations  are 
necessary  to  ensure  that  the 
content,  form,  and  sequencing  of 
the  instructional  presentations 
will  eventually  be  able  to  satisfy 
the  instructional  objectives  of  the 
courseware  developer. 

The  methodology  provides  tools  to 
acquire  the  knowledge  needed  in  each 
module  and  defines  an  architecture  to 
integrate  the  contributions  provided  by 
the  different  experts  who  make  up  the 
authoring  team.  The  concept  of  an 
Intelligent  Authoring/Instructional 
System,  lA/IS,  is  introduced  in  this 
research.  Figure  1  shows  that  an  lA/IS 
fits  in  the  intersection  of  the  three 
disciplines  of  software  engineering, 
artificial  intelligence,  and  education. 


Figure  1.  Relationship  between  AI,  software 

engineering,  and  education. 


3.  A  Model  for  an  Intelligent 
Authoring/Instructional  System 
(lAAS) 

In  the  model  that  has  been  developed 
[KERN90],  the  knowledge  that  is 
acquired  from  the  authoring  team  is 
stored  in  three  knowledge-bases  that  co- 
operate in  determining  the  interactive 
instructional  materials  that  are 
presented  to  the  student.  The  authoring 
team  consists  of  a  content  expert,  an 
instructional  design  expert,  and  a 
courseware  developer.  This  team  uses 
acquisition  tools  provided  by  the  lA/IS 
to  gather  the  required  knowledge.  This 
insures  that  the  data  is  stored  in  a  form 
that  can  be  accessed  by  the  other  co- 
operating components  of  the  system. 

3,1  Two  Modes  of  Operation 

Figure  2  shows  the  data  flow  diagram  for 
the  system  in  authoring  mode.  The 
acquisition  tools  are  used  to  extract  the 
required  knowledge  from  the  members 
of  the  authoring  team.     Each  of  the 
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The  methodology  consist  of  the 
following  steps: 

lAcquire  a  prescriptive  instructional 
theory  based  on  theoretical 
considerations  from  an 
instructional  designer.  This  is 
accomplished  with  the  use  of  the 
Prescriptive  Theory  Acquisition 
Screen  that  is  part  of  the 
Instructional  Strategy  Acquisition 
Tool.  An  instructional  strategy  is 
prescribed  [LAND83]  based  on  the 
subject  to  be  presented,  the  type  of 
learning  desired,  and  a  description 
of  the  intended  student 
population, 

ZAcquire  a  description  of  the 
targeted  student  population  from 
the  courseware  developer.  The 
Course  Design  Acquisition  Tool 
contains  a  Student  Parameter 
Acquisition  Screen  to  obtain  this 
information.  The  parameters 
describe  the  student  population  in 
terms  of  (i)  previous  knowledge  of 
the  content  area,  (ii)  general 
aptitude,  (iii)  how  meaningful  the 
content  is,  and  (iv)  interest  in 
learning.  A  Learner  Control  Metric 
is  calculated  from  these  factors.  The 
higher  this  value  [STEI89],  the  more 
control  the  student  has  to  determine 
the  sequencing  oi  the  presentations 
and  the  rate  of  progress  through  the 
course. 

3.  Acquire  a  priority  sequence  for 
searching  the  indices  of  the  case- 
based  instructional  strategy  system 
so  the  system  can  access  the  ''closest 
match''  to  the  current  sub-objective. 
There  is  a  Priority  Acquisition 
Screen   in   the   Course  Design 


Acquisition  Tool  to  set  these 
priorities.  When  an  exact  match  is 
not  available,  the  author  can  set  the 
priority  of  the  following  factors;  (i) 
complexity  of  the  subject  matter,  (ii) 
type  of  learning  required,  or  (iii) 
unique  qualities  of  the  student 
population.  The  highest  priority 
will  determine  the  primary  search 
key,  then  the  secondary  search  key, 
and,  finally,  the  tertiary  search  key. 

^Acquire  the  performance  objectives 
for  the  instructional  system  from 
the  courseware  developer.  The 
Course  Design  Acquisition  Tool, 
also,  contains  an  Objective 
Acquisition  Screen  for  acquiring 
these  performance  objectives. 

5Sub-divide  the  course  objectives 
into  sub-objectives  until  at  the 
lowest  level  each  objective  is  just 
one  concept  or  skill  that  can  be 
presented  to  the  student  as  one  unit 
of  instruction  [GAGN«8].  The  Sub- 
Objective  Acquisition  Screen  of  the 
Course  Design  Acquisition  Tool  is 
used  to  accomplish  this. 

6.Retrieve  the  ''closest  match"  to 
suggest  an  instructional  plan  to  the 
courseware  developer.  There  are 
three  Plan  Requirement  Screens 
that  are  part  of  the  Course  Design 
Acquisition  Tool  to  describe  the 
current  instructional  plan  in  terms 
of  subject,  type  of  learning,  and 
student  characteristics.  The  search 
priorities  set  in  step  3  wiU  be  used  to 
direct  the  search  through  the 
instructional  material  that  has  been 
previously  developed. 


Figuie  2.  Data  flow  diagram  for  the  lA/IS  in 
authoring  mode. 


experts  is  queried  and  given  advise  as  his 
knowledge  is  extracted  for  use  in  the 
lA/IS.  The  knowledge  that  is  extracted  is 
refined  until  it  is  in  a  form  that  can  be 
incorporated  into  one  of  the  knowledge- 
bases. 

In  instructional  mode,  the  Learning 
Monitor  supervises  the  sequencing  of 
the  instructional  presentations  to  the 
student.  Figure  3  shows  the  flow  of  data 
during  this  instructional  mode.  The 
student's  interaction  with  the  system  is 
used  by  the  Learning  Monitor  to  gather 


statistics  about  the  student's  knowledge. 
Periodically,  the  statistics  are  used  to 
update  the  parameters  in  the  model  of 
the  student  and  compare  the  student's 
knowledge  structure  to  the  content 
knowledge  structure  created  by  the 
content  expert.  Attention  is  paid,  not 
only,  to  the  student's  knowledge  as 
evaluated  from  tests,  but  also,  to  the 
pattern  of  references  to  other  related 
presentations  to  infer  the  relationships 
the  <^tudent  is  making  between  the 
different  "chunks"  of  content  knowledge 
he  is  attempting  to  learn, 

4,  Steps  in  the  Tool-Supported 
Authoring  Methodology 

The  tool-supported  methodology  to 
author  rn  ITS  [KERN92]  consists  of  ten 
steps  to  acquire  the  knowledge  required 
by  the  system.  Refer  to  Figure  2  to  follow 
tlie  data  flow  as  the  system  is  created. 


Content 
Knowledge 
Base 


student 
Model 


Case-Based 
Instructional 
Strategy 
Systenn 


Learning 
Monitor 


Student 


Figure  3.  Data  flow  diagram  for  the  lA/lS  iit 
instructional  mode. 
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7 Modify  the  prescribed  instructional 
plan  to  satisfy  the  current  objective. 
The  Instructional  Strategy 
Acquisition  Tool  contains  an 
Instructional  Plan  Modification 
Screen  to  specify  these  changes.  The 
author  can  reuse  as  much  of  the 
previously  developed  instructional 
material  as  he  deems  appropriate 
and  make  modifications  as 
necessary. 

SAcquire  the  content  material  for 
the  presentation  screens  required  for 
each  of  these  sub-objectives  from 
the  content  expert.  The  Content 
Acquisition  Tool  includes  a 
Template  Selection  Screen  and  five 
different  Template  Screens  from 
which  to  choose.  The  templates 
include  the  following  types  of 
instructional  displays  (i)  present 
information,  (ii)  provide  an 
interactive  demonstration,  (iii)  drill, 
(iv)  question,  and  (v)  test. 

9Switch  to  instructional  mode  when 
the  instructional  materials  for  all  of 
the  course  objectives  have  been 
created.  At  this  point  the  emphasis 
changes  from  authoring  to 
instr'icting. 

lO.Evaluate  the  instructional 
materials  and  make  appropriate 
modifications  if  defects  are  revealed. 
If  the  instructional  materials  are 
found  to  be  defective,  the  author  can 
return  to  steps  6,  7,  and  8  and  make 
changes  to  the  instructional  strategy 
and/or  the  instructional  materials. 

This  is  an  iterative,  recursive 
methodology.  Step  5  is  done  recursively 
until  you  reach  the  lowest  level  sub- 


objective  for  which  the  instructional 
materials  are  created.  Then,  steps  6,  7, 
and  8  form  an  iterative  loop  that  is 
repeated  to  create  the  instructional 
materials  for  each  of  the  sub-objectives 
specified  in  step  5.  If  step  10  reveals  a 
defect  in  the  system,  it  is  corrected  by 
repeating  ^  oop  containing  steps  6,  7, 
and  8.  The  author  is  finished  when  he  is 
satisfied  with  the  instructional 
effectiveness  of  the  tutoring  system  that 
has  been  created. 

5.  Evaluating  an  ITS 
In  order  to  be  judge  educationally 
effective  as  a  tutoring  system,  the  lA/IS 
must  satisfy  the  goals  of  four  sets  of 
actors;  the  content  experts,  the 
instructional  design  experts,  the 
courseware  developers,  and  tiie  students. 
To  be  considered  intelligent  [BURN881, 
the  instructional  system  must  pass  the 
following  tests  for  intelligence.  The 
instructional  system  must  (i)  "know"  the 
domain  knowledge  well  enough  to  draw 
inferences  and  to  solve  problems,  (ii)  be 
able  to  assess  the  student's  level  of 
mastery  of  the  domain  knowledge,  and 
(iii)  adapt  its  instructional  strategy  to 
reduce  the  differences  between  the 
student's  performance  and  the 
embedded  content  expert's  performance. 

The  lA/IS  addresses  these  issues  with 
the  following  AI  techniques: 

•  The  content  knowledge  is  stored  in 
Cognitive  Associative  Structures 
(CAS).  This  is  a  hypermedia 
structure  that  emphasizes  the 
relationships  that  exists  between 
"chunks"  of  the  content  area.  A 
CAS  has  the  capacity  to  store  both 
declarative     and  procedural 


knowledge.  The  Authoring  Team 
can  include  all  the  knowledge  that  is 
necessary  to  draw  inferences  and 
solve  problems. 

•A  model  of  the  student's  knowledge 
is  stored  in  a  student  version  of  the 
content  CAS.  This  CAS  is 
initialized  using  parameters 
supplied  by  the  Courseware 
Developer  about  ability  level  and 
prerequisite  knowledge  of  the 
intended  student  population.  As 
the  student  interacts  with  the 
instructioiial  system,  changes  in  his 
knowledge  structure  are  inferred 
from  his  responses,  and  the  model 
is  adjusted  accordingly.  Parameters 
are  maintained  to  evaluate,  not 
only,  the  student's  mastery  of  the 
content  "chunks",  but,  also,  to  infer 
the  relationships  that  are  being 
strengthen  by  the  pattern  of 
references  to  related  "chunks"  of 
knowledge. 

•  Differential  modeling  techniques 
are  used  to  indicate  differences 
between  the  expert's  version  of  the 
CAS  for  the  content  knowledge  and 
the  student's  version  of  the  CAS 
[WILK88],  A  performance-oriented 
comparison  would  give  a  problem 
to  the  student  and  then  compare  the 
method  the  student  uses  to  solve 
the  problem  to  the  method  specified 
by  the  content  expert.  In  contrast  to 
this  approach,  a  knowledge-oriented 
comparison  compares  the  structure 
of  the  studeiit's  knowledge  to  the 
structure  of  the  content  knowledge 
specified  by  the  content  expert.  This 
lA/IS  takes  the  knowledge-oriented 
approach  because  in  the  long  run, 
performance  will  be  more  correct  if 


the  knowledge  structure  is  carefully 
developed.  In  Van  Lehn's  study  of 
subtraction  errors  made  by  school 
children  IVANL88],  he  found  that 
"bugs"  are  caused  by  correct 
execution  of  an  incorrect  procedure. 
This  supports  the  conclusion  that 
faulty  knowledge  is  the  most  likely 
problem,  not  faulty  performance. 

•  a  case-based  instructional  strategy 
knowledge-base  is  consulted  to 
improve  the  instructional 
effectiveness  ol  the  system  when 
defects  are  revealed  [HAMM89], 
[HAMM91]. 

6.  Criteria  for  an  authoring  system 

An  authoring  system  must  be  more  than 
just  a  high-level  programming 
environment  and  a  graphics  editor 
IMERR85].  It  must,  also,  offer  assistance 
in  the  instructional  design  process.  The 
lA/IS  includes  a  case-based  instructional 
strategy  system  (C-BISS)  that  is 
initialized  with  generic  prescriptive 
instructional  strategies.  The  C-BISS 
suggests  a  strategy  to  the  author  based  on 
the  description  of  the  intended  student 
population  and  the  type  of  learning  that 
is  desired.  The  author  is  free  to  make 
modifications  to  the  suggested  strategy. 
These  modified  prescriptions  are  then 
added  to  the  C-BISS. 

The  lA/IS,  also,  contains  basic 
instructional  templates.  These  include  a 
template  for  each  of  the  following 
instructional  methods:  (i)  presentation 
of  information,  (ii)  interactive 
demonstration,  (iii)  drill,  (iv)  question, 
and  (v)  test.  The  author  can  modify 
these  templates  and  add  new  templates 
as  necessary. 
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7.  A  Prototype  lA/IS:  HOBIE 

I  have  built  a  prototype  lA/IS,  called 
HOBIE  (Hypermedia  Object-Based 
Intelligent  Educator),  to  demonstrate  this 
methodology  and  system  architecture. 
The  tools  to  extract  the  knowledge  from 
the  authoring  team  were  developed  on 
an  Apple  Macintosh  SE  computer  using 
HyperCard  as  a  graphics  editor  and 
programming  environment. 

The  instructional  system  was  created 
using  my  tool- supported  methodology. 
The  curriculum  was  designed  as  an 
introductory  component  for  an 
undergraduate  course  in  assembler 
language.  The  content  area  is  an 
introduction  to  the  binary  and 
hexadecimal  numbering  systems 
including  two's  complement  notation 
and  data  representation  formats.  It 
includes  presentation  of  new  material, 
interactive  demonstrations  of  the 
conversion  algorithms,  drill,  and  testing 
of  the  student's  knowledge. 

8,  Conclusions 

The  principles  of  software  engineering 
[GHEZ91]  must  be  applied  to  the  process 
of  creating  complex  software  systems. 
The  development  of  a  methodology  is 
instrumental  in  ensuring  the  inclusion 
of  these  principles  in  order  to  promote 
the  successful  development  of  an 
effective  ITS.  The  principles  of 
separation  of  concerns,  modularity, 
abstraction,  anticipation  of  change,  and 
incrementality  are  incorporated  into  the 
ten  step  tool-supported  methodology 
introduced  in  this  paper. 
The  software  qualities  of  user- 
friendliness,  reusability,  and  evolvability 
have  also  been  provided  for  in  the  lA/IS 
in  the  following  ways.  Tlie  use  of  well- 


developed  tools  and  a  graphics  editor 
increase  the  user-friendliness  of  the 
system.  The  author  can  recall  similar 
sub-objectives  that  have  been  satisfied  in 
the  past,  modify  the  previously 
developed  instructional  material  and 
reuse  the  modified  version  to  satisfy  the 
new  sub-objective.  Tne  ability  to  switch 
back  into  authoring  mode  whenever  the 
instructional  mode  reveals  a  deficit  in 
the  instructional  system,  allows  the 
instructional  system  to  evolve  into  a 
more  effective  system  as  it  is  being  used. 

This  methodology  offers  assistance  in 
the  authoring  process  and  will 
encourage  the  creation  of  complete 
instructional  systems.  As  more  systems 
are  available  for  use,  it  will  be  possible  to 
empirically  validate  the  instructional 
strategies  that  are  most  effective  in  a 
computer-assisted  learning  environ- 
m:)nt. 
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Abstract 

This  paper  describes  the  results  of  thjree  separate  experimental  evaluations  of  an  intelligent  tutoring 
system  (ITS)  called  Modified  Advanced  Learning  for  MSE  ( MALM ),  This  system,  designed  to 
teach  troubleshooting  and  operations  procedures  to  military  conmmucation  personnel,  was 
systematically  altered  in  order  to  examine  the  impact  of  specific  ITS  capabilities.  Specifically,  we 
focussed  on  tlie  systems  advice  function  and  error  feedback  function.  In  order  to  understand  the 
impact  of  these  experiments,  we  review  past  evaluations,  provide  a  brief  description  of  MALM,  ofid 
detail  methods  and  results  of  the  three  experiments. 


This  paper  presents  the  results  of  an  experiment  with  Modified  Advanced  Leaminjg  for  MSE 
(MALM),  an  Intelligent  Tutoring  System  (ITS)  for  teaching  troubleshooting  and  operations 
procedures  for  Mobile  Subscribe  Equipment  (MSE,  a  communication  network  used  by  the  Army). 
We  begin  the  description  of  these  experiments  with  a  brief  discussion  of  evaluations  that  have  been 
conducted  on  ITSs.  This  discussion  will  focus  on  both  the  results  and  methods.  In  addition,  in 
order  to  understand  any  generalizations  that  might  be  drawn  from  the  results  of  this  collection  of 
experiments,  the  ITS  used  will  be  described  in  more  detail. 

After  twenty  years  of  ITS  research,  it  is  surprising  how  little  effort  has  been  given  toward 
evaluation.  Perhaps  this  is  because  of  the  old  AI  maxim,  "If  it  works,  then  it  is  NOT  AI."  Perhaps 
it  is  because  results  of  evaluations  are  less  than  expected  and  researchers  are  hesitant  to  place  these 
results  out  for  public  consumption.  Perhaps  there  is  littie  money  or  motivation  after  large  scale 
development  efforts  to  conduct  any  formal  evaluations.  Whatever  the  reasons,  there  are  surprisingly 
few  evaluations  reported.  One  of  the  most  complete  reviews  was  written  by  Legree  and  Gillis 
(1991).  In  this  paper,  they  conclude  that  the  effect  size  of  ITSs  is  approximately  1.0  relative  to 
lecture  approaches  to  instruction  and  human  tutoring.  This  result  is  true  if  the  ITS  coverage  is  large 
enough  (in  terms  of  time)  to  have  an  effect,  whereas  small  scale  evaluations  (short  treatments)  tend  to 
indicate  no  differences. 

To  begin  this  review,  we  will  describe  the  large  scale  evaluations.  Kurland,  Granville,  and 
MacLaughlin  (1990)  conducted  a  large  scale  evaluation  of  the  MACH  in  ITS  that  was  used  for 
Army  training.  This  ITS  was  considered  large  scale  because  the  content  covered  32  hours  of 
material.  The  metl^od  of  evaluation  was  to  compare  a  group  of  soldiers  that  learned  fix)m 
"traditional"  instruction  with  a  group  of  soldiers  that  learned  from  the  MACH  in  tutor.  The  results 
of  this  evaluation  indicate  that  the  MACH  m  tutor  has  an  effect  size  of  1.0  relative  to  regular  class 
instruction.  Anderson,  Boyle,  and  Reiser  (1985)  evaluated  their  Lisp  tutor  by  using  both  regular 
classro'^ni  instruction  and  the  ITS.  In  addition,  they  included  a  human  tutor  group.  Their  results 
also  int    .te  about  a  1.0  effect  size  and  the  treatment  lasted  between  1 1.4  an(^  ^-6.5  hours.  Chrey 
(1991)  -.ed  a  similar  evaluation  strategy  to  evaluate  his  whole-number  subtraction  tutor.  The  effect 
size  was  a  little  less  than  1 .0  and  the  treatment  was  4.5  and  6.2  hours.  Raghavan  and  Katz  (1989) 


517 

430 


compared  a  group  who  had  access  to  their  economics  tutor  (Smithtown)  with  a  group  who  did  not 
have  access.  Not  surprisingly,  the  ITS  group  took  less  than  half  the  time  to  complete  assignments. 
Legrce  and  Gillis  (1991)  suggest  that  the  best  way  to  perform  evaluations  of  ITSs  is  io  compare  the 
system  to  lecture  and  human  tuioring.  This  notion,  while  suggested  by  others  (e.g.,  Anderson,  et 
al.,  1985),  is  based  on  Bloom's  (1984)  2-Sigma  Problem. 

According  to  this  view,  the  average  child  who  is  individually  tutored,  will  score  2  standard 
deviations  higher  on  achievement  tests  than  the  average  child  iti  a  lecture-based  class.  The  problem 
is  to  find  a  cost  effective  alternative  to  one-on-one  tutoring.  In  fact,  some  have  suggested  that  this  is 
a  goal  of  rrS  research  and  development  (Anderson  &  Reiser,  1985).  One  of  the  results  of  looking  at 
evaluation  from  this  perspective  is  that  others  attempt  to  make  effect  size  estimates.  Whereas  tlie 
above  research  would  indicate  approximately  1.0  for  an  effect  size  for  ITS,  Kulik  and  Kulik  (1987) 
would  suggest  a  0.31  effect  size  for  CAI.  Others  would  reduce  this  figure  to  approximately  zero 
(Qark,  1985).  These  studies  have  focused  on  broad  measures  of  impact  on  achievement  As  has 
been  mentioned  elsewhere  (Orey,  1991),  Clark  (1983)  and  Reeves  (1986)  have  described  many 
dangers  in  using  these  ill-defmed  groups  for  comparison  purposes.  However,  we  have  chosen  to 
use  similar  groups  for  the  fu^t  two  experiments  described  in  this  paper.  We  do  not  use  this  approach 
in  the  third  experiment.  We  chose  to  use  a  more  detailed  analysis  for  this  experiment.  The  reason 
for  this  is  thatLegree  and  Gillis  (1991)  suggest  that  the  smaller  the  treatment,  the  less  likely  to  show 
differences.  We  suggest  that  the  gross  measures  may  be  a  contributing  factor.  Therefore,  examining 
a  more  refined  approach  would  ci'eate  the  possibility  for  finding  any  differences  that  might  exist. 

The  architecture  of  MALM  was  based  on  a  problem  solving  perspective  and  it  makes  use  of 
some  of  the  latest  computer-based  instructional  technologies:  ITS,  learning  environments,  and 
hypermedia.  These  techniques  are  combined  so  that  problem  solving  in  a  MSE  context  can  be 
explored  in  the  somewhat  safe  and  inexpensive  environment  of  an  IBM  PC  computer. 

MALM  was  designed  for  high  bandwidth  diagnosis  using  a  model  tracing  technique.  It  was 
implemented  in  a  hypermedia  environment  that  allowed  the  learners  to  explore  the  environment, 
much  like  that  in  a  learning  environment  The  knowledge  base  representation  combines  a  frame 
based  representation  and  a  rule  based  implementation.  Problems  are  generated  using  an  automated 
parsing  process  and  instruction  is  provided  when  errors  and  help  occur.  These  aspects  of  MALM 
were  added  (for  a  more  cx)mplete  description  of  MALM,  please  refer  to  Orey,  et  al.,  (1992))  to  the 
already  existing  system  that  was  developed  by  Galaxy  Corporation  (Coonan,  Johnson,  Nortcm,  & 
Sanders,  1990).  We  also  removed  the  advice  and  error  feedback  and  created  a  version  which 
presenteed  the  entire  solution  process  on  one  screen.  The  learners  needed  to  determine  what  they 
had  and  had  not  done.  This  linear  list  of  steps  became  our  CAWike  Linear  Advice  (LA)  version.  If 
we  take  Kulik  and  Kulik  (1987)  at  face  value,  then  the  effect  size  estimate  for  the  LA  version  of 
MALM  would  actually  be  slighdy  higher  than  0.31.  Therefore,  the  comparisons  between  groups 
would  be  harder  to  detect  differences  because  they  are  closer  to  same  effect  size. 

Methodology 

Introduction 

Regardless  of  the  differences  in  prior  knowledge  between  the  available  subjects  at  the 
University  of  Georgia  and  the  target  users  that  MALM  was  designed  for,  the  purpose  of  this 
evaluation  was  to  determine  the  effectiveness  of  both  error  feedback  and  the  advice  function  in 
MALM.  In  the  formative  evaluation  conducted  in  tlie  Spring  of  1991  at  the  University  of  Georgia, 
frequent  access  to  advice  was  observed  during  the  process  of  solving  problems  among  subjects  (see 
Orey,  et  al.,  1992).  It  was  assumed  that  the  advice  function  played  an  essential  role  in  helping 
participants  without  content  knowledge  to  solve  problems  in  MALM.  Because  of  this  reliance  on  die 
advice  function,  the  summative  evaluation  primarily  focussed  on  the  effect  of  the  advice  function  and 
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error  feedback  function  in  MALM.   In  order  to  examine  the  effect  of  the  advice  function,  a  detailed 
analysis  was  perfoimed.  It  compared  ITA  with  LA.  The  specifics  of  the  experiraent  are  described 
below. 

Materials 

Essentially,  there  were  two  versions  of  MALM  that  were  created  for  this  evaluation,  the  ITA 
and  LA  versions.  Of  all  the  problems  that  were  created,  we  chose  to  examine  only  one 
troubleshooting  problem.  The  cwnputei"  would  then  collect  the  needed  information  for  the  analysis. 
It  saved  every  cUck  of  the  mouse  and  every  keayboard  entry. 

Participants/Groups 

The  participants  were  10  undergraduate  studciuS  in  Elementary  or  Early  Childhood  Education 
and  graduate  students  in  Computer-Based  Education  or  Instructional  Technology  at  the  University  of 
Georgia.  These  participants  wen.  split  into  two  equal  groups  of  5  participants. 

There  were  two  groups  in  the  experiment  The  ITA  group  had  access  to  an  advice  function  that 
adapts  to  the  learner  based  on  actions  they  perform.  In  addition,  this  group  had  the  diagnostic 
component  of  MALM  to  help  them  to  learn  finom  their  errors.  These  aspects  of  MALM  were  deemed 
the  most  "intelligent"  components  of  the  system  and  were  only  available  to  the  ITA  group  during  the 
experiment 

The  LA  group  in  tliese  experiments  also  used  only  the  computer  for  learning.  However,  they 
did  not  have  "intelligent"  advice  or  error  diagnosis.  Instead,  the  LA  group  was  provided  a  screen 
Iisun£  ail  the  steps  for  correctly  carrying  out  the  procedures  required  for  troubleshooting  in  MSE. 
Instead  of  corrective  feedback  on  errors  made,  the  system  informed  them  when  an  error  occurred 
(not  what  the  error  was  and  why  it  was  incorrect). 

Procedure 

The  procedure  for  the  experiment  follows  a  strategy  adapted  from  Allison  an^  Liker  (1982)  and 
described  in  Legree,  Gillis  and  Orey  (in  press).  This  strategy  used  a  lag  sequential  analysis.  !n  this 
procedure,  the  intent  is  to  detenmne  the  probability  of  accessing  advice  or  making  an  error  one,  two, 
or  more  steps  after  accessing  advice  die  first  time.  More  generally,  what  is  the  impact  of  the 
computer's  behavior  (via  messages)  on  the  behavior  of  the  learner's  first,  second,  etc.  behavior 
(interaction)  following  the  computer's  behavior.  In  order  to  perform  this  analysis,  MALM  was 
modified  so  that  dribble  files  would  be  created  that  would  save  every  step  that  was  pexfoimed  by 
learners  as  they  solved  the  problem.  In  addition,  a  transparent  button  was  placed  in  the  top  middle  of 
each  screen  so  that  the  experimenter  could  indicate  when  human  advice  or  feedback  was  provided. 
In  this  way,  the  impact  of  the  advice  and  enor  functions  could  more  directly  be  assessed. 

The  procedure  used  was  to  have  each  participant  solve  one  troubleshooting  S'jbscriber 
problem.  Participants  were  randomly  assigned  to  one  of  two  groups.  A  minimal  description  of  the 
system  was  provided.  The  participant  was  then  asked  to  solve  the  problem.  Whenever  the 
participants  arrived  at  an  impasse,  they  would  ask  for  assistance  from  the  experimenter.  The 
experimenter  would  provide  one  declarative  statement  as  help.  If  this  was  not  enough,  the 
participant  could  ask  again.  Each  access  to  help  was  recorded  in  the  dribble  file  via  the  transparent 
button. 

Analysis 

We  used  a  lag  sequent  lal  probability  procedure  (Allison  &  L:ker,  1982).  This  is  a  social 
interaction  analysis  which  Allison  and  Liker  describe  in  terms  of  behaviors  of  married  couples.  In 
otlier  words,  does  the  negative  behavior  of  spouse  one  affect  the  behavior  (positive  or  negative)  of 
spouse  two  at  different  behavioral  lags.  For  our  purposes,  there  are  two  important  characteristics. 
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First,  the  interaction  analysis  is  done  between  two  behavior  generating  entities  (for  them  two 
spouses;  for  us-a  computer  and  a  learner).  Second,  the  behaviors  are  categorized  as  dichotomous 
(this  is  not  a  requirement  according  to  Allison  and  Liker,  but  it  wotJcs  for  us). 

The  dichotomous  data  for  Allison  and  Liker  was  positive  and  negative.  We  chose  to  catcgcnize 
-  learner  behaviors  as  c<»Tect  or  incorrect.  Actual  possible  behaviors  of  the  learners  could  be 
categorisDcd  as  1)  performing  a  correct  problem  solving  action;  2)  navigating  through  the  system;  3) 
accessing  infOTmation;  4)  seeking  advice;  and,  5)  perfOTning  an  incorrect  problem  solving 
procedure.  We  then  classified  behaviors  1  through  3  as  ccsrcct  and  behaviors  4  and  5  as  incorrect 
Obviously,  the  first  behavior  should  be  considered  aMrrect  and  the  fifth  incorrect  The  other  three 
could  be  debatable.  However,  we  ccmsidered  that  actively  using  the  system  was  normal  problem 
solving  behavicH*  (navigating  and  accessing  information).  Giciang  on  the  Help  button,  on  the  other 
hand,  we  considered  incorrect  because  we  felt  that  this  was  one  of  the  two  ways  to  resolve  an 
impasse.  The  otlier  way  was  to  commit  an  enror.  Using  this  classification  scheme,  the  analysis  was 
produced. 

The  next  step  in  the  analysis  was  to  consider  which  of  the  computer's  behaviors  were 
impwtant  analysis.  Because  we  considered  advice  and  error  feedback  to  be  the  most  "intelligent" 
components  of  the  system,  these  were  included  in  the  analysis.  In  addition,  we  chose  to  examine  the 
impact  of  human  advice.  These  behaviors  need  to  be  compared  with  some  base  behaviors.  Because 
the  computer  does  not  respond  following  a  coitect  action,  we  chose  to  compare  the  other  behaviors 
with  those  following  a  correct  step. 

We  chose  to  examine  learner  behaviors  from  one  to  ten  behaviors  following  the  computer 
behavior.  A  2  X  2  contingency  table  was  generated  for  each  lag.  One  dimension  of  this  table  was 
whether  the  learner  perfonned  a  correct  or  incorrect  behavior.  The  other  dimension  was  the 
computer's  behavior.  In  the  computer  advice  analysis,  these  two  behaviors  were  computer  advice  ot 
conect  slep»  (which  actually  was  no  behavior).  Therefore,  there  were  ten  2  X  2  contingency  tables 
for  computer  advice  behavior  versus  no  behavior,  ten  for  error  feedback  behavior  versus  no 
behavior,  and  ten  for  human  advice  feedback  versus  no  feedback.  This  yields  three  analyses 
(computer  advice,  error  feedback,  and  human  advice).  In  addition,  there  were  two  groups  (TIA  and 
LA).  Each  analysis  generates  ten  2  X  2  contingency  tables.  Each  group  was  analyzed  using  the 
three  analyses.  The  result  was  that  there  were  2  groups  times  3  analyses  tin>es  10  contingency 
tables,  or  sixty  2X2  tables. 

The  first  research  question  to  be  addressed  is:  Is  the  learner's  behavior  different  following  the 
computer's  behavior  (or  human  tutor)  tiian  foUowing  no  behavior  at  lags  1  tiirough  10?  In  order  to 
answer  this  question,  chi-squares  were  calculated  for  each  contingency  table.  An  alpha  level  of  .05 
was  used.  This  was  done  for  all  sixty  tables  each  with  its  own  question. 

The  second  research  issue  was  to  consider  whether  leanier  behavior  patterns  differed  for  the 
LA  and  ITA  groups.  This  analysis  was  performed  at  each  of  the  thirty  points  (the  three  categories 
[computer  advice,  error  feedback,  and  human  advice]  times  the  ten  lags).  Allison  and  Liker  (1982) 
suggest  a  procedure  for  calculating  a  F-score  for  comparisons  between  groups.  This  procedures  was 
applied  to  Uiis  evaluation  and  an  alpha  level  of  .05  for  two-tailed  tests  was  used  ' 

Results  and  Discussion 

We  attempted  to  examine  the  impact  of  advice  and  errors  on  leamer^s  behaviors.  In  addition, 
we  examined  the  difference  between  learner  behaviors  when  using  the  ITA  version  versus  the  LA 
version.  To  begin  tiiis  analysis,  refer  to  Table  1  for  tiie  overall  numbers.  The  pattern  of  results  here 
are  similar  to  those  in  the  learning  phase  of  Experiments  1  and  2.  That  is,  the  ITA  group  (range 
10-29)  used  the  advice  function  moj-e  frequently  than  the  LA  group  (range  6-11).  In  addition,  the 
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number  of  errors  that  occurred  in  the  two  groups  were  about  equal  (TTA  range  1-26;  LA  range  1-23). 
These  two  results  are  similar  to  the  results  found  in  the  first  two  experiments.  However,  Table  1 
also  includes  the  number  of  time  the  learner  asked  the  experimenter  for  help.  We  did  not  keep  track 
of  human  advice  in  the  other  experiments  and  it  appears  that  the  LA  group  needed  that  fomi  of  help 
much  more  frequently,  almost  2.5  times  moie  frequently. 

As  suggested  earlier,  the  advice  fiinctic«i  in  the  LA  version  of  MALM  may  not  be  very  useful. 
The  experimenter  frequendy  mentioned  that  the  LA  participants  would  not  be  able  to  complete  any 
problems  without  experimenter  assistance.  This  fact  would  render  the  LA  version  useless  in  a 
remote  implementation  of  the  software  (no  one  to  ask  for  help). 

The  next  part  of  the  experiment  involves  the  results  of  the  sixty  contingency  tables  (see  Table 
2).  We  will  begin  with  the  impact  of  computer  advice.  First,  look  at  the  lags  of  1  and  2  for  the  FTA 
group  in  Figure  1.  The  gap  between  the  two  lines  would  suggest  that  there  is  something  different  in 
die  two  events.  The  quantitative  analysis  bears  this  out  Both  of  these  are  statistically  significant  at 
.01  level  (c^  =  7.33,  df =1,  at  lag  1).  This  result  would  indicate  tliat  the  leamer^s  behavior  following 
computer  help  is  different  from  when  the  computer  does  not  do  anything.  Further,  the  numt^ 
would  indicate  that  the  learner  tends  to  do  something  correct  following  computer  help  (58  times 
correct  out  of  79  at  lag  1)  whereas  the  opposite  occurs  following  an  event  where  the  computer  does 
noihmg  (8  times  out  of  34  at  lag  1).  The  pattern  is  also  evident  at  lag  2  (54  times  out  of  79  and  14 
times  out  of  34,  respectively).  Although  there  arc  some  other  gaps  between  die  two  lines,  none 
prove  to  be  statistically  significant 


Table  1.  Means  and  standard  deviations  (in  parentheses)  for  accesses  to  cofnputer  advice,  human 
advice  and  total  number  of  errors  for  the  five  (5)  participants  in  each  group. 


Groups 


Type  of  Interaction 

ITA 

LA 

Computer  Advice 

15.8  (7.6) 

7.4  (2.1) 

Error  Feedback 

13.8  (12.0) 

11.4  (9.5) 

Human  Advice 

4.2  (2.2) 

1.04(4.3) 

Note:  ITA=Intelligent  Advice,  and  LA=Linear  Advice 


Next,  refer  to  Figure  1  again,  but  this  time  focus  on  the  LA  group.  Here  again  a  lag  of  1 
proves  to  be  significant  ( c^.  =7.76,  df=l).  This  result  also  indicates  tiiat  learners  using  the  LA 
version  tend  to  do  something  correct  following  computer  advice  (26  times  out  of  35)  aiKi  tend  to  do 
something  incorrect  following  a  correct  step  (20  times  out  of  34).  However,  it  does  not  persist  to  a 
lag  of  2.  There  is  another  significant  lag  at  8  ( c^  =1 1 .20,  df=l).  In  this  case,  the  learner  is  more 
likely  to  make  an  enor  8  steps  after  accessing  help  (18  times  out  of  34)  and  wove  likely  to  do 
sometiung  correct  8  steps  after  perfomiing  a  correct  step  (21  times  out  of  24).  We  arc  not  sure  what 
tills  means.  Our  impression  after  seeuig  several  people  use  the  different  versions  is  tiiat  advice  does 
not  have  any  effect  8  steps  following  accessing  it  (and  18  times  out  of  32  is  close  to  a  50-50  chance). 
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However,  why  did  learners  using  the  LA  tend  to  do  something  correct  8  steps  foUov/ing  ac^ss  to 
advice? 

The  next  issue  to  consider  regarding  Figure  1  is  whether  the  pattern  of  results  differs  between 
the  ITA  and  LA  groups.  Z-scores  were  calculated,  as  suggested  by  Aliiscm  and  Uker  (1982),  for 
each  lag  and  appear  in  Table  2.  The  critical  value  at  the  .01  alpha  level  is  1.96.  As  can  be  seen, 
only  lags  of  8  and  9  exceed  this  value  and  what  this  means  is  not  at  all  clear.  At  lag  8,  the  fact  that 
LA  learners  tended  to  perform  a  conrect  step  8  steps  following  a  correct  step  (88%  of  the  time) 
explains  why  the  test  statistic  was  significant  especially  since  the  other  possihOities  at  lag  8  are  at 
about  50  percent  correct.  However,  what  this  implies  about  the  effects  of  the  LA  versus  ITA  version 
is  not  at     clear.  Ten  lag  behaviors  were  included  for  no  apparent  reason.  Now  that  there  were 
significant  differences  at  lags  8  and  9,  it  is  not  clear  how  to  interpret  this.  At  lag  9,  it  is  even  mcne 
confusing.  Seventy-two  percent  of  the  behaviors  9  steps  following  a  correct  step  were  correct  for 
the  ITA  group  whereas  Scventy-CHie  percent  of  behaviors  9  steps  follow  advice  were  COTcea  for  the 
LA  group.  TTiis  disparity  accounts  for  the  significant  Z-score.  Again,  what  this  means  for  actual 
differences  terween  ITA  and  LA  is  not  clear. 


Table  2.  The  chi-squared  values  for  the  sixty  different  lag  possibilities  and  the 
Z'Score  difference  measure  between  groups. 


Groups 


Computer  Advice  Error  Feedback  Human  Advice 


Lag 

ITA 

LA 

Z-Score 

ITA 

LA 

Z-Score 

ITA 

LA 

Z-Score 

1 

24.35** 

7.76** 

1.10 

6.93** 

1.56 

0.99 

4.78* 

8.37** 

0.07 

2 

7.33** 

0.12 

1.41 

0.05 

1.32 

1.00 

0.80 

1.82 

0.08 

3 

3.17 

0.04 

0.94 

0.19 

223 

0.80 

1.28 

0.68 

0.32 

4 

0.67 

3.12 

0.89 

0.00 

056 

0.42 

0.55 

0.03 

0.46 

5 

0.38 

2.83 

0.94 

0.18 

0.03 

0.41 

2.09 

0.40 

0.63 

6 

0.28 

0.44 

0.17 

0.00 

0.00 

0.03 

0.74 

0.00 

0.68 

7 

0.04 

0.34 

0.31 

0.94 

0.13 

0.91 

0.18 

0.05 

0.16 

8 

0.07 

11.20** 

2.74** 

0.26 

7.35**  1.81* 

0.05 

3.76 

157 

9 

2.29 

2.00 

2.04** 

3.35 

0.75 

1.88* 

1.11 

0.27 

1.14 

10 

2.55 

0.27 

L41 

0.71 

2.49 

1.72* 

0.01 

3.98* 

1.19 

** 

p<0.05 
p<0.01 
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The  other  "intelligent"  aspect  of  MALM  that  we  exaitiined  was  its  ability  to  provide  appropriate 
feedback.  The  LA  version  only  provided  the  fact  that  an  error  occurred  whereas  the  ITA  version 
explained  why  the  error  was  incorrect,  what  the  error  was,  and  recommended  correction.  In  Figure 
2,  the  results  on  error  feedback  for  the  ITA  and  LA  groups  is  shown.  The  first  result  to  discuss  is 
the  lag  of  1  for  the  ITA  group.  This  result  was  statistically  significant  (refer  to  Table  2).  However, 
rather  than  tending  to  do  something  correct  following  error  feedback  (34  times  out  of  69  for  lag  1), 
the  effect  is  really  manifest  in  the  fact  that  learners  tended  to  do  something  incorrect  following  a 
correct  step  (26  errors  out  of  24  dmes  at  lag  1).  The  remaining  9  lags  seem  to  be  closely  related  (the 
lines  nearly  overlap  in  Figure  2  except  at  the  most  significant  lag  of  9). 


The  second  point  about  tlie  results  is  in  regard  to  the  LA  group.  There  is  no  significant 
difference  between  behaviors  following  correct  and  erroneous  performance  until  lag  8.  Like  the 
results  of  computer  advice  described  above,  this  effect  is  largely  due  to  the  fact  that  learners  in  the 
LA  group  tended  to  do  something  correct  8  behaviors  after  perfonning  a  correct  step.  Why  this 
occurred  is  ambiguous.  The  remaining  lag  behaviors  tend  to  be  similar. 

The  third  aspect  of  the  results  to  be  examined  is  whether  there  exists  a  difference  between 
behaviors  related  to  error  feedback  for  the  two  groups.  Once  again,  statistical  significance  was  not 
indicated  until  lags  8, 9,  and  10.  And,  as  described  for  computer  advice,  it  is  unclear  what  this  result 
might  mean.  A  statistical  explanation  might  suggest  that  by  chance  alone,  some  of  the  thirty  Z-scores 
calculated  might  be  significant  (5  significant  Z-scores  out  of  30  calculated  is  more  than  a  chance 
level). 

As  a  final  comment  about  errors,  thei^  seemed  to  be  a  grcat  difference  for  the  ITA  group 
between  performance  following  computer  help  and  correct  steps  at  lags  1  and  2  of  Figure  1.  This 
disparity  seems  to  be  as  great  as  the  disparity  found  in  the  LA  group  portion  of  Figure  1,  the 
disparity  found  in  the  T'^A  group  portion  of  of  Figure  2,  and  the  disparity  found  in  the  LA  group 
portion  of  Figure  2.  \v ;  would  suggest  here  that  this  is  the  most  powerful  result  of  the  analysis  to 
this  point  in  favor  of  intelligent  tutoring  systems.  That  is,  computer  help  has  an  impact  on 
performance  for  lags  1  and  2  for  the  ITA  group.  This  impact  is  not  nearly  as  great  for  computer  help 
for  the  LA  group  nor  is  it  that  great  for  error  feedback  for  either  group.  This  result  might  suggest 
that  "intelligent"  help  may  be  a  useful  featiu^  to  include  in  a  system.  Error  feedback  does  not,  on  the 
other  hand,  seem  to  have  much  of  an  impact  on  learner  performance. 

As  described  above,  human  advice  was  not  only  invaluable,  it  may  have  been  essential  for  the 
LA  learners.  How  this  advice  impacted  on  the  lag  behaviors  will  be  described  here.  Referring  to 
Figure  3,  die  ITA  group  had  only  one  significant  difference  between  behaviors  following  human 
advice  (1 1  correct  out  of  21)  and  those  following  correct  performance  (8  correct  out  of  34)  and  that 
was  at  a  lag  of  1  (refer  to  Table  2  for  list  of  chi-squared  values).  Again,  this  result  is  primarily  due 
to  the  fact  that  learner's  in  the  ITA  group  tended  to  perform  an  incorrect  step  following  a  correct  step. 

The  LA  group  had  two  significant  lags,  one  at  lag  1  and  the  other  at  lag  10.  At  lag  10,  the  LA 
learners  tended  to  do  something  correct  following  a  correct  step  (15  out  of  22)  and  something 
incorrect  following  human  advice  (26  errors  out  of  45).  As  with  the  odier  analyses,  it  is  not  clear 
how  to  interpret  this  difference.  However,  the  difference  at  lag  1  would  suggest  that  human  advice 
had  a  differential  effect  on  the  LA  group's  performance.  In  fact,  the  perfoimance  of  the  LA  group 
following  human  advice  (37  correct  out  of  51)  is  much  better  than  performance  following  a  correct 
step  (14  correct  out  of  34).  Further,  the  performance  following  human  advice  for  the  LA  group  at 
lag  1  is  similar  to  the  performance  of  learners  following  computer  advice  at  lag  1  for  both  tihe  LA  and 
ITA  groups  (73%,  74%,  and  73%,  respectively).  However,  this  was  not  true  of  the  ITA  group 
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receiving  human  advice  (52%). 

A  possible  explanation  for  this  result  may  be  that  the  type  of  information  that  was  asked  for  by 
the  ITA  learners  may  be  different  than  the  type  of  information  requested  by  the  LA  learners,  ot 
provided  by  either  version  of  MALM.  MALM  provided  pnxedural  information.  This  analysis  has 
examined  procedural  perfcHmance.  Perhaps,  the  LA  learners  used  the  experimenter  to  access  noore 
detailed  pix)cedural  iniormation  than  that  provided  by  the  LA  version  of  MALM.  The  ITA  learners, 
on  the  other  hand,  may  have  used  the  experinienter  to  access  concepnial,  declarative,  or  some  other 
form  of  knowledge.  There  is,  however,  no  evidence  to  substantiate  this  interpretation.  It  should  be 
noted  that  there  was  no  statistical  difference  between  the  ITA  and  LA  groups  on  the  hirnian  advice 
part  of  the  analysis  (see  Table  2). 

Conclusions 

In  initiial  experiments,  we  found  that  learners  in  the  ITA  group  used  the  help  function  more 
frequently  than  the  learners  in  the  other  groups.  We  designed  this  experiment  to  mo^e  closely 
examine  what  occurs  when  learners  are  using  tine  help  system.  In  addition,  we  examined  the  error 
diagnostic  component  and  the  learners*  need  to  ask  for  more  help  from  the  experimenter. 

Computer  advice  seemed  to  have  had  more  of  an  impact  on  learners  in  the  ITA  group  than  the 
LA  group  at  lags  1  and  2.  As  described  above,  this  may  be  due  to  a  lower  performance  percentage 
following  a  correct  step.  However,  this  lower  percentage  may  be  due  to  the  power  of  the  advice 
function.  By  the  end  of  the  session,  many  ITA  learners*  interactions  followed  the  pattern 
computerAdvice-navigarion-  correctStep-computerAdvice.  This  pattern  is  not  surprising.  The 
advice  messages  in  the  ETA  version  were  stated  generally  as,  "The  next  appropriate  step  is  to  do  X. 
To  do  this,  you  need  to  access  device  Y  ..."  Therefore,  the  learners  accessed  advice,  navigated 
through  the  system  to  the  appropriate  device,  performed  the  correct  step,  and  accessed  advice  again. 
This  pattern  did  not  occur  with  the  LA  group.  In  fact,  the  LA  group  exhibited  no  common  patterns 
until  the  very  end  of  the  problem.  At  this  point,  it  was  common  that  the  learner  followed  the  pattern 
"  computerAdvice-navigation-correctStep  -navigation-conectStep".  This  pattern  is  not  surprising 
either.  The  computer  advice  for  the  LA  group  listed  all  the  correct  steps  to  solve  the  problem  on  one 
screen.  By  the  end  of  the  problem,  the  learners  had  become  sufficiendy  knowledgeable  of  the 
system  to  be  able  to  use  this  advice.  The  conclusion  to  be  drawn  from  this  result  is  that  the  ITA 
advice  is  more  helpful  to  learners  who  have  less  knowledge.  Once  they  have  acquired  a  working 
knowledge  of  the  system,  the  LA  function  may  be  sufficient  for  the  learners.  However,  more 
detailed  advice  may  still  be  needed  when  learners  begin  to  solve  slighdy  different  problem  within  the 
domain. 

Errors  feedback,  on  the  other  hand,  does  not  seem  to  have  much  of  an  impact.  This  is  of 
concern  given  that  ITS  designers  spend  a  great  deal  of  time  in  the  design  and  acquisition  of 
information  to  give  capture  errors  and  provide  feedback.  Referring  to  Figure  2,  the  LA  group  was 
performing  at  about  50  percent  correct  step  level  following  feedback  and  following  a  correct  step. 
The  ITA  group  had  a  significant  difference  at  this  first  lag.  This  difference  is  due  to  the  relatively 
poor  performance  following  a  correct  step.  Following  error  feedback,  the  ITA  participants 
performed  at  about  the  50  percent  level.  There  can  be  a  variety  of  conclusions  to  draw  from  this. 
One  is  that  MALM's  error  feedback  mechanisms  were  of  a  poor  quality,  and  these  results  cannot  be 
generalized  to  all  systems.  Another  would  be  that  the  content  under  study  does  not  lend  itself  to 
using  error  feedback.  Still  another  is  tiiat  error  diagnosis  is  a  waste  of  time  and  should  not  be 
continued.  Or,  finally,  another  would  be  that  the  error  feedback  did  have  an  effect  on  learners,  but 
our  measures  were  not  sensitive  to  the  differences.  Regardless  of  the  explanation,  there  ought  to  be 
more  detailed  analyses  of  the  impact  of  error  feedback  on  learners  using  ITSs.  This  functionality  is 
one  of  the  most  cosdy  components  of  an  ITS  and  we  are  having  difficulty  finding  effects. 
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We  kept  track  of  human  advice  and  participants  in  the  LA  group  used  hunian  advice  much  more 
frequendy  than  the  ITA  group.  The  experimenters  agreed  that  the  participants  in  the  LA  group  would 
not  have  been  able  to  do  anything  if  the  experimenter  did  not  intervene.  The  ITA  version  could  be 
sent  to  people  who  do  not  have  access  to  human  advice  and  they  could  solve  problems  within 
MALM.  Tlie  LA  version  could  not.  This  makes  the  ITA  version  worth  doing  without  even 
examining  the  other  aspects  of  the  system.  Therefore,  in  the  final  analysis,  if  someone  was 
considering  the  idea  of  developing  an  ITS,  tlie  results  of  these  experiments  tend  to  support  this 
decision  with  a  few  qualifications. 

As  a  final  point  to  this  study,  we  recommend  the  use  of  the  methodology  used  in  this 
experiment.  The  analysis  technique  was  based  on  the  an  analysis  technique  for  the  study  of 
human-human  interactions.  We  have  modified  it  to  analyze  human-ITS  interaction.  Others  could 
modify  it  to  gamine  any  human-computer  interaction.  In  later  papers,  we  will  examine  how  to  use 
this  methodology  to  study  the  interaction  patterns  of  learners  using  hypermedia  programs. 
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Figure  1.  The  analysis  of  computer  help  for  the  two  groups. 
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Figure  2.  The  analysis  of  computer  error  feedback  for  the  two  groups* 
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Figure  3.  The  analysis  of  human  help  for  the  two  groups. 
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Research  and  CBI  in  Business:  The  Need  for  Dialog 

Gary  C  Powell^  MA. 
Thomas  C  ReeveSy  Ph.D. 

. ^  study  determined  if  and  how  research  in  Computer-Based  Instmction  is 
transferred  to  business  and  industrial  settings  and  practices;  and  if  the  results  of 
those  practices  (*Veal  world"  applications)  come  back  to  the  researchers  in  the 
universities.  Interviews  (a  qualitative  data  collection  method)  were  conducted  with 
two  groups:  researchers  (professors)  of  Instructional  Design,  and  instmctional 
design  professionals/practitioners  in  large  corporate  settings.  A  general  inteiview 
guided  approach  was  utilized  to  elicit  responses  to  questions  pertaining  to  the  nature 
of  educational  research  and  its  influence  on  the  business  practice  of  CBI  and 
Instructional  Systems  Design  (ISD).  Findings  indicate  two  disparate  views 
(academe  vs.  business/industry)  on  the  subject.  Specifically,  from  the  perspective  of 
business  and  industiy,  educational  research  does  transfer  to  them  in  a  number  of 
ways.  The  feedback  to  the  researchers  is  lacking  however,  and  is  mainly  their 
blame.  From  the  view  of  the  practitioners,  the  researchers  need  to  be  more 
proactive,  and  less  arrogant.  While  business  and  industry  feel  that  the  research 
going  on  in  the  universities  is  not  very  focused  in  on  their  immediate  future  needs,  it 
is  valuable,  and  should  continue  to  be  supported  even  more  so  than  it  is  now. 
University  researchers  should  take  more  risks,  be  more  tuned  in  to  the  needs  of 
practice,  and  take  advantage  of  the  fact  that  they  can  fail  with  little  repercussions. 
From  the  perspective  of  the  university  researchers,  it  seems  that  research  as  a  whole 
is  not  meant  to  be  all  applied  and  directly  transferable;  but  rather  a  contributor  to  a 
knowledge  base  of  general  theories,  models,  principles,  and  hypotheses. 
Unfortunately,  much  of  it  is  poorly  conceived,  and  makes  the  transfer  process  that 
much  more  difficult.  Of  tliat  research  which  is  substantive  and  applicable  to 
business  practice,  the  researchers  are  of  the  opinion  that  only  a  handful  of 
companies  use  it.  And  only  a  select  few  of  those  that  do  use  it  make  any  effort  to 
give  feedback  (via  presentations,  publications,  etc.)  to  the  field  as  a  whole.  From 
the  viewpoint  of  the  researchers,  business  and  industry  needs  to  be  more  aware  of 
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the  purposes  and  limitations  of  research,  and  not  assume  that  there  will  always  be 
some  sort  of  direct  transfer  or  application, 

littroductm  and  review  of  liter mr^ 


This  study  is  designed  to  determine  if  and  how  research  in  Computer-Based 
Instruction  is  transferred  to  business  and  industrial  settings  and  practices;  and  if  the 
results  of  those  practices  ("real  world"  applications)  come  back  to  the  researchers  in 
the  universities. 

To  provide  a  context  for  understanding  the  nature  of  this  study,  a  literature 
search  was  performed  to  determine  the  status  quo  regarding  the  possible  transfer  of 
educational  research  to  business  and  industry,  and  the  relationships  between 
universities  and  corporations.  Tlie  following  sections  deal  with  the  issues  of 
educational  research  transfer  to  education  systems,  technical  research  transfer  to 
business  and  industry,  and  university  partnerships  with  business  and  industry  for 
educational  reasons. 

Educational  research  transfer  to  educational  systems 

Educational  research  does  have  a  direct  influence  on  policy-makers  and 
practitioners  in  our  educational  system.  While  the  relationship  between  educational 
research  and  educational  practice  ~  and  even  the  contributions  of  social  science 
research  as  a  whole  -  has  often  been  tenuous  and  uncertain,  its  does  exist. 

For  instance,  Shavelson  (1988)  argues  that  educational  research  contributes  to 
policy  and  practice  via  three  avenues:  by  helping  construct,  by  challenging,  and  by 
changing  the  way  policy-makers  and  practitioners  view  particular  problems. 
Shavelson  states: 

I  have  seen  how,  for  example,  research  on  teaching  has  changed 
the  way  practitioners  and  policy-makers  view  teaching,  how 
research  on  military  recmiting  in  community  colleges  has 
challenged  policy-makers'  notions  about  current  recruiting  efforts 
and  their  potential  payoff,  and  how  school  and  teacher 
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effectiveness  research  has  changed  the  way  poHcy-makers  think 
about,  collect,  and  interpret  statistics  (indicators)  oa  the  "health" 
of  education  (p.  4). 

Unfortunately  however,  these  influences  on  practitioners  and  policy-makers  is  not 
"frequent  enough  or  as  effective  as  it  might  be." 

Confusion  abounds  as  to  the  role  educational  research  should  play.  Should  it 
directly  and  immediately  apply  to  a  particular  issue,  problem,  or  decision;  or  should 
it  produce  knowledge,  which  allows  policy-makers  to  make  rational  actions  based 
on  this  knowledge,  which  in  tum  will  lead  to  "good"  education  which  benefits  all 
groups  in  society  uniformly  gjid  equally?  It  seems  that  either  option  is  unreasonable, 
and  sets  up  misleading  expectations. 

Nevertheless,  most  people  would  agree  that  as  education  is  an  applied  field, 
educational  research,  in  the  aggregate,  should  lead  to  improvements  in  the  practice 
of  schooling.  This  is  the  expectation  of  most  legislators  and  foundations  who 
provide  the  bulk  of  the  resources  for  educational  research.  In  particular,  these 
supporters  expect  improvement  in  school  achievement,  especially  among  those  "at 
risk"  (Slavin,  1990). 

Slavin  (1990)  argues: 

One  of  the  most  important  reasons  that  the  impact  of  research  on 
education  seems  so  haphazard  is  that  education  research  is 
primarily  engaged  in  a  search  for  variables  that  may  relate  to 
improved  outcomes... 
Widescale  change  in  education  practice,  however,  will  come  about  via  the 
implementation  of  models.  Models  are  instructional  or  school  organizational 
programs  composed  of  many  individual  variables  which  may  not  have  come  from 
research  (Slavin,  1990).  While  some  models  (e.g.  cooperative  leaming,  mastery 
learning)  do  have  a  conceptual  basis,  education  research  on  models  is  lacking 
because  the  academic  reward  system  favors  research  which  builds  theory  through 
variables.  Slavin  suggests  that  when  models  such  as  cooperative  leaming  are 
published  in  the  select  journals,  it  is  only  because  they  speak  to  well-defined  and 
easily  identified  bodies  of  theory. 

Technical  research  transfer  to  business  and  industry 
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There's  a  uew  way  for  industry  to  tap  the  most  sophisticated  research  in  the 
world:  that  performed  in  universities.  Joint  ventures  with  universities  make  sense 
for  both  parties:  corporations  acquire  exclusive  access  to  powerful  research  and 
development  (R&D),  while  the  universities  gain  a  new  source  for  research  support. 

Most  collaborations  between  industry  and  universities  are  in  fields  such  as 
biotechnology  (not  education  or  computer-based  instruction).  These  collaborations 
can  take  a  number  of  forms,  such  as  partnerships,  joint  ventures,  and  the  formation 
of  neo-public  companies.  This  marriage  between  commerce  and  academe  is 
typically  for  the  procreation  of  commercially  exploitable  and  profitable  products. 
Examples  include  Monsanto *s  multi-million  dollar  joint  venture  with  Washington 
University  (St.  Louis),  and  Hoechst  A.G/s  support  of  the  molecular  biology 
department  at  MIT  (Bartlett,  1988). 

The  number  of  companies  tuming  to  universities  for  research  is  growing.  One 
reason  is  that  there  tends  to  be  more  innovation  in  university  research  than  ia 
industry.  Although  non-academic  employees  bring  much  to  their  workplace,  they 
tend  to  lose  some  creativity  over  time.  Another  reason  university  research  has  a 
special  appeal,  is  that  universities  provide  a  strong  research  base,  which  many 
corporations  lack  due  to  a  shortage  of  technical  professionals  (Aaron,  1988). 

Tapping  into  university  research  can  take  several  forms.  Unlike  the  formal 
processes  mentioned  previously  (e.g.,  joint  ventures),  some  industries  are  less 
formal  in  their  approach.  For  instance,  a  company  R&D  executive  may  read  a 
professor's  paper,  contact  her  or  him,  and  proceed  from  there.  Attending  academic 
and  consortium  meetings,  conferences,  and  symposiums;  and  reading  journals, 
studies,  and  papers  are  also  ways  of  discovering  new  technologies  and  theories. 
And  for  those  who  are  "strapped  for  time'*  and  can  not  attend  conferences  and  the 
like,  there  are  companies  like  University  Science  Partners  Inc.  that  act  as  agents  to 
bring  university  technology  to  the  commercial  sector  (Aaron,  1988). 

Universities  have  been  also  been  quite  proactive  in  forming  relationships  with 
business  and  industry.  Technology  transfer  from  universities  to  business  is 
becoming  big  business  for  academics.  Many  schools  have  an  office  of  technology 
transfer,  and  some,  like  the  University  of  California,  have  two.  The  University  of 
Washington  holds  science  and  technology  exnositions  -  featuring  the  latest  in 
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biotechnology,  sensing  technology,  biomedical  instrumentation,  and  image 
processing  -  solely  to  attract  venture  capitalists  (Aaron,  1988). 

It  is  quite  clear  that  this  country  is  in  a  competitive  struggle  with  other 
industrial  countries  for  markets  in  advanced  technologies.  The  United  States  has 
not  realized  the  full  potential  of  these  technologies,  and  the  university-industry 
relationship  is  part  of  that  une^ploited  potential. 

University  partnerships  with  business  and  industry  for  educational  reasons 

Wliile  not  as  not  as  strong  in  magnitude  or  number,  there  do  exist 
collaborations  and  interfaces  between  business  and  industry  and  educational 
institutions  and  foundations  in  the  area  of  educational  technology. 

One  example  is  a  unique  partnership  that  was  forged  in  the  Santa  Clara 
Valley  of  Northem  California  for  the  shared  use  of  a  telecommunications  delivery 
system  to  provide  a  cost  effective  means  of  providing  telecommunications  delivered 
education  and  training  to  company  work  sites  (Capell,  Dermody,  Maier,  &  Monfort, 
1986).  The  unique  partnership  brought  together  a  group  of  companies  for  the 
purpose  of  sharing  services  and  resources  that  are  of  mutual  benefit.  Through  the 
efforts  of  a  non-profit  organization,  the  Industry  Education  Council  of  Santa  Clara 
County,  a  partnership  model  for  sharing  telecommunication  resources  was 
established  that  could  lead  tlie  way  to  business/industry/public  sector  partnerships 
throughout  the  country.  The  innovative  partnership  in  Silicon  Valley  proved  to  be 
very  successful  in  providing  industry  with  highly  effective  but  low  cost  instruction  in 
the  workplace.  This  example  is  but  one  that  demonstrates  that  diverse  entities  can 
work  together  effectively  for  the  mutual  benefit  of  all  conceme^.i 

Other  university-corporate  collaborations  are  like  the  ones  found  at  Arthur 
Andersen  &  Co.  and  IBM.  Arthur  Anderson  uses  the  research  and  staff  of  the 
Institute  for  the  Learning  Sciences  located  at  Northwestern  University  (Evanstan, 
II).  This  group,  headed  by  Roger  Schank,  provides  research  findings,  models  and 
theories  to  Arthur  Andersen.  Additionally,  one  Professional  Education  Division 
instructional  designer  attends  the  Institute  as  an  "intern"  each  year.  While  there,  the 
person  works  toward  a  master^s  degree. 
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IBM's  ISD  Competency  Center  (a  part  of  IBM's  US  Education  division)  has 
an  Academic  Council  comprised  of  six  noted  professors  in  Instructional  Systems 
Design.  They  serve  as  a  combination  think  tank  and  review  board.  For  instance, 
they  assist  in  setting  education  agendas,  and  review  high  level  design  documents. 

While  these  "joint  ventures"  have  no  commercial  appeal  like  those  in  other 
fields  like  biotechnology,  tliere  is  clearly  a  transfer  of  educational  research  and 
expertise  going  on. 

Another  area  where  education  can  be  improved  through  partnerships  with 
business  and  industry  is  that  of  education  reform  (Theede,  1985).  Our  American 
educational  system  desperately  needs  to  improve  and  respond  to  the  impending 
needs  of  change.  In  order  to  avoid  becoming  a  "Nation  at  Risk,"  a  reorganization  of 
our  educational  system  is  necessary.  This  reorganization  must  provide  for  greater 
networking  between  education  and  business  and  industry.  While  coordination 
between  business  and  industry  and  education  is  not  new,  it  will  require  that  new  role 
responsibilities  be  defined  so  desired  outcomes  are  realized.  In  order  to  achieve 
excellence  in  education,  there  needs  to  be  greater  corporate  participation  of  business 
and  iiidustry  in  the  universities'  educational  preparation  programs. 

A  shared  philosophy  between  business  and  industry  and  education  must 
contain  a  functional  relationship  based  on  serving  the  needs  of  each  partner  through 
wise  planning  and  use  of  available  resources  to  produce  mutually  beneficial 
outcomes.  In  order  to  meet  these  challenges,  Theede  suggests  a  cyclical  interaction 
and  reciprocity  process:  a)  Establish  philosophical  base,  b)  Set  partnership  category 
(fiscal  support,  material/equipment,  personnel  resources,  staff  development, 
research  &  development,  or  new  inforaiation),  c)  Identify  general  goals,  d) 
Determine  role  responsibilities,  e)  Action  plan  is  developed,  f)  Action  plan  is 
implemented,  g)  Evaluation  (start  back  with  'a'). 

What  does  this  all  mean?  From  the  university  perspective,  it  means  than 
improvement  requires  change.  Secondly,  the  charge  for  improvement,  accountability 
and  excellence  is  established  at  the  university  level.  And  lastly,  the  excellence 
needed  to  improve  education  will  not  happen  v/ithout  a  greater  partnership  with 
business  and  industry. 
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Methodology 

A  qualitative  approach  was  used  to  determine  if  and  how  research  in 
Computer-Based  Instruction  is  transferred  to  business  and  industrial  settings  and 
practices;  and  if  the  results  of  practice  come  back  to  the  researchers  in  the 
universities.  Specifically,  the  methods  used  were  interviews. 

Two  different  sets  of  questions  were  used  in  this  study:  one  for  university 
professors/researchers  and  one  for  practitioners  in  a  business  setting.  Responses  to 
the  business  setting  form  were  elicited  via  a  focus  group  interview.  The 
participants,  who  were  all  in  the  practice  of  instmctional  technology  in  a  large 
accounting  firm,  were  as  follows:  a  director  of  a  technology  group,  a  manager  in  the 
same  technology  group,  an  experienced  senior  instructional  designer,  and  a 
principal/associate  director  of  the  same  technology  group. 

Responses  to  the  professor/researcher  form  were  elicited  via  a  general 
interview  guide  approach.  One  professor  of  instmctional  technology  was 
interviewed  face-to-face.  The  individuals  were  not  randomly  selected,  but  were 
chosen  based  on  the  personal  relationship  the  investigator  has  with  them. 

R?suU^!;  Business  and  Industry 


What  is  the  role  of  university  educational  research  as  you  see  it? 

 As  an  ISD  person  in  doing  CBT,  I  don't  use  it.  Research  from  the  fields  of 

Human  Factors  and  Cognitive  Psychology  is  more  appropriate  in  understanding 
interactivity  issues.  My  goal  is  to  assure  that  people  learn  the  material  so  they  can 
perform.  The  key  issue  is  learning  the  material,  not  using  the  software,  or  how  to 
use  it.  I  can  recall  a  piece  of  educational  software  that  contained  a  20  minute 
lead-in  just  on  how  to  use  it, 

 Research  from  journalism  and  advertising  are  helpful  as  far  as  page  layout  and 

screen  layout  are  concerned.  Research  from  text  design  is  helpful  as  well. 
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 Academic  research  should  not  be  constrained  to  academic  research  for  corporate 

needs.  They  should  not  be  limited.  The  challenge  we  face  in  business  and  industry 
is  to  cheiTy  pick  the  research,  that  is,  to  get  the  approach  research  for  our  needs.  Its 
our  responsibility. 

 There  is  a  real  disconnect  between  what  corporations  do,  and  what  it  is  that 

researchers  research. 

 Some  research  transfers,  some  is  ignored. 

How  should  it  contribute  to  educational  practice  in  business  and  industry? 

 Educational  research  should  take  more  risks  if  it  has  the  proper  and  adequate 

funding.  Academe  has  less  pressure  to  complete  a  project.  Its  no  big  deal  to  start 
something,  say  its  doesn't  work,  and  start  over  again  in  an  academic  setting. 
Academe  can  afford  risk  taking  that  corporations  can't. 

 Researchers  in  universities  can  start  a  project  over  again  from  scratch. 

Corporations  can't.  Universities  can  do  research,  so  do  it!  Industrial  settings  are 
more  constrained  by  money. 

 Our  role  is  to  get  the  research,  see  what  works,  and  tell  the  rest  of  our 

colleagues  what  to  apply. 

 Researchers  should  produce  research  which  makes  the  professional  more 

productive  and  give  them  more  leverage. 

 Coiporate  support  of  research  is  not  brand  new.  A  lot  comes  out  of  the  Dept.  of 

Defense  and  Dartmoth  University. 

Does  educational  research  transfer  to  your  organization's  practice?  If  so  how; 
if  not,  why? 

 Yes  it  transfers. 

 Conferences.  Our  staff  is  highly  encouraged  to  join  professional  associations 

and  go  to  the  conferences.  When  they  retum,  they  are  encouraged  to  write  a  paper 


Page  9 


to  distribute  to  their  department.  Also:  technology  forums  with  key  speakers, 
recruiting  of  ISD'ers  with  Ph.D/s  and  MA's.,  personal  contacts,  competitors,  other 
universities,  joumals,  vendors  of  technology. 

Describe  its  impact  (if  so). 

 There  certainly  is  an  impact.  But  its  not  obvious.  There  is  company  inertia;  the 

effects  don't  appear  over  night.  Educational  research  is  just  a  small  part  of  the 
variance  in  how  we  do  things. 

How  do  you  suggest  that  more  educational  research  gets  to  you  as 
practitioners? 

 More  research  on  what  it  is  we're  doing.  What  is  being  researched  now  has 

very  low  pay  offs  in  the  next  five  to  ten  years. 

 But  just  because  it  has  no  immediate  financial  payoff,  doesn't  mean  its 

meaningless. 

 There  is  a  10  year  cycle  from  something  to  go  from  tiie  research  lab  into 

practice. 

Do  you  feel  that  the  relationship  between  your  organization  and  university 
researchers  in  instructional  technology  is  adequate? 

 Its  probably  never  adequate  because  unless  you're  in  contact  with  them  on  a 

monthly  basis,  you  have  to  wait  for  an  aiticle  to  be  published,  which  is  usually  a 
year  after  the  person  wrote  it. 

 Joumals  do  act  as  a  filter  though. 

 Personal  contacts  also  act  as  a  filter  too. 

Do  you  feed  the  results  of  putting  research  and  theories  into  practice  back  to 
the  researchers  in  the  universities?  Why  or  why  not? 
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 No.  Unless  researchers  and  academics  come  out  and  see  what  we  are  doing. 

Also,  our  interns  go  back  and  talk  about  whaf  s  going  on. 

 We  don't  want  to  share  everything.  We  keep  things  that  work  well  for  us 

private. 

 Plus  business  and  industry  is  much  further  ahead  than  what  the  universities  are 

doing.  Some  companies  for  instance  have  to  take  a  4  year  engineering  student  [one 
with  a  B  A]  and  train  him  for  a  year  before  he's  useful.  So  in  some  respects,  that 
makes  us  in  competition  with  junior  colleges. 

 It's  a  matter  of  competitive  advantage. 

 Those  that  come  and  see  what  we're  doing  though,  can  see  it  all.  Most  simply 

go  "ooh  ah.'^ 

 There's  a  lack  of  interest  for  universities  to  get  feedback  in  some  cases.  They 

can  be  arrogant  at  times.  For  instance,  one  academic  called  me  once  so  we  could 
get  together  so  he  could  tell  me  what  we  can  do  here  based  on  his  research.  If  he 
only  knew.  Some  universities  also  have  a  distaste  for  stuff  from  the  private  sector. 

 Sometimes  we  do  call  the  authors  of  articles  we  read  in  the  journals.  Plus  we 

know  a  lot  of  people  in  the  field. 

 Money  plays  a  significant  role  in  facilitating  the  communication  process.  The 

feedback  process  is  not  as  simple  as  picking  up  the  phone. 

Any  other  comments? 

 More  transfer! 

 Universities  should  continue  to  do  research,  and  not  cut  back.  Universities  need 

more  federal  dollars.  You  are  valued. 


Results:  University 
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What  is  the  role  of  university  educational  research  as  you  see  it? 

 Discovery  of  new  knowledge.  Theory  and  model  building,  some  is  action 

orientated;  a  continuum  of  purpose. 

 Contributing  to  the  knowledge  base.  However,  some  research  is  better  than 

others.  We  don't  do  a  good  job  of  theory  building,  but  an  ok  job  on  model  building. 

 Only  2  or  3  names  come  to  mind  in  our  field  in  terms  of  model  building. 

 Basically  we  are  poor  in  theory  building,  reasonable  at  model  building,  and  do 

veiy  little  action  orientated  research. 

How  should  it  contribute  to  educational  practice  in  business  and  industry? 

 Different  ways.  There  is  an  expectation  that  it  all  will  transfer,  which  is 

unfounded.  There  is  not  always  a  direct  application  of  what  we  do.  Research 
provides  alternative  explanations.  For  instance,  if  a  child  can  not  read,  it  isn't 
necessarily  because  he's  stupid.  Research  contributed  to  what  we  know  about 
dyslexia. 

 Research  points  out  complexities  but  not  necessarily  prescriptions.  But  its  still 

worth  doing. 

 There's  a  quality  problem  with  research.  Some  is  poorly  conceived  and  is 

nonsense  and  therefore  should  not  effect  practice. 

 Direct  appHcations  of  research  will  always  be  limited.  Conditions  always  vary  in 

the  field,  and  you  can't  always  match  them.  To  put  the  results  of  educational 
research  to  use,  I  suggest  collecting  a  body  of  things  which  we  can  make 
generalities  with. 

 An  effect  is  there,  but  it  isn't  direct.  Research  provides  general  principles, 

general  hypotheses,  etc. 
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Do  you  feel  that  educational  research  transfers  to  business  settings?  If  so, 
how;  If  not,  why  not? 

 Some  places,  to  some  degree.  At  places  like  Arthur  Andersen  it  does* 

 The  transfer  takes  place  via  people  in  industry  who  have  received  some  formal 

training  in  research.  They  keep  up  with  what's  going  on  via  journals,  conferences, 
etc. 

 The  military  for  instance  has  an  active  research  program. 

Describe  its  impact  (if  so). 

 Fairly  limited.  A  lot  of  places  don't  get  research  findings  in  a  systematic  way. 

How  do  you  suggest  that  educational  research  gets  to  those  practitioners  who 
aren't  getting  it? 

 They  need  to  realize  that  what  they  want  from  educational  research,  they  aren*t 

going  to  get.  We  need  to  communicate  what  the  realistic  results  of  research  are. 
Plus  we  need  to  do  a  better  job  of  interpreting  what  we  do.  Practitioners  in  business 
and  industry  need  to  think  in  research  terms  -  how  it  (research)  can  be  used  and  its 
limitations, 

 We  need  to  do  a  better  job  of  communicating  the  power  and  limitations  of 

research.  We  also  need  to  cut  the  trivial  stuff.  It  isn't  as  simple  as  cutting  the 
jargon. 

Do  you  feel  that  the  relationship  between  you  as  a  researcher  and  the  business 
practice  of  instructional  design  is  adequate? 

 The  relationship  could  be  enhanced.  We  need  to  stop  blaming  them  though. 

Do  you  feel  the  results  of  putting  research  and  theories  into  practice  is  fed  back 
to  the  researchers  in  the  universities?  Why  or  why  not? 
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 To  some  extent.  Some  folks  in  business  and  industry  do  write  up  what  they  do 

and  go  to  conferences  to  present  it.  Some,  like  the  military,  do  not  encourage 
dissemination. 

 It  varies  with  corporations.  The  IBM's,  Apple's,  and  Arthur  Andersen's  do  send 

their  people  to  conferences.  They  are  the  exceptions.  The  others  need  to  make  an 
effort. 

Any  other  comments? 

 A  lot  of  places  could  benefit  from  what  we  already  know,  but  we  don't  know  a 

hell  of  a  lot.  The  research  base  is  not  tliat  big,  but  better  use  could  be  made  of  it. 


Discussion 


It  seems  clear,  from  the  perspective  of  business  and  industry,  that  educational 
research  does  transfer  to  them  in  a  number  of  ways.  The  feedback  to  the 
researchers  is  lacking  however,  and  is  mainly  the  blame  of  the  researchers.  They 
need  to  be  more  proactive,  and  less  arrogant.  While  business  and  industry  feels  that 
the  research  going  on  in  the  universities  is  not  very  focused  in  on  their  immediate 
future  needs,  it  is  valuable,  and  should  continue  even  more  so  than  it  is  now. 
University  researchers  should  take  more  risks,  be  more  tuned  in  to  the  needs  of 
practice,  and  take  advantage  of  the  fact  that  they  can  fail  with  little  repercussions. 

From  the  perspective  of  the  university  researchers,  it  seems  that  research  as  a 
whole  is  not  meant  to  be  all  applied  and  directly  transferable;  but  rather  a 
contributor  to  a  knowledge  base  of  general  theories,  models,  principals,  and 
hypotheses.  Unfortunately,  much  of  it  is  poorly  conceived,  and  makes  the  transfer 
process  that  much  more  difficult.  Of  that  research  which  is  substantive  and 
applicable  to  business  practice,  only  a  handful  of  companies  use  it.  And  only  a 
select  few  of  those  that  do  use  it,  make  any  effort  at  giving  the  field  any  feedback 
(via  presentations,  etc.).  Basically,  business  and  industry  needs  to  be  more  aware  of 
the  purposes  and  limitations  of  research,  and  not  assume  that  there  will  always  be 
some  sort  of  direct  transfer  or  application. 
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Conclusion 


It  is  hoped  that  this  study  will  spark  more  probing  in  the  area  of 
Computer-Based  Instruction  research  transfer,  as  more  is  needed.  It  is  clear  that 
practitioners  in  business  and  industry,  and  researchers  in  universities,  do  not  see 
'*eye-to-eye"  on  a  number  of  issues.  The  possibility  of  a  conference  of  researchers 
and  practitioners  on  this  topic  of  research  transfer  and  feedback  is  a  very  good  idea 
necessary  it  would  seem  -  and  has  caught  the  interest  of  many  people  already. 
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What  L.  S.  Has  Learned 


Steven  D.  Tripp 
University  of  Kansas 


Abstract 

LS  has  taught  business  statistics  at  the  University  of  Kansas  for 
26  years.  He  has  received  many  awards  for  his  teaching  and  is  now  a 
Chancellor's  Club  Teaching  Professor,  This  paper  is  a  distillation  of 
a  two-hour  interview  with  him  concerning  his  methods  and 
experiences. 

Introduction 

LS  is  University  of  Kansas  Chancellor's  Club  Teaching 
Professor  of  business  statistics.  He  has  received  many  awards 
during  his  26  years  of  teaching  and  is  frequently  requested  to  lecture 
on  his  use  of  technology  in  teaching.  After  seemg  one  such  lecture,  I 
approached  him  about  doing  an  interview  concerning  his  teaching 
methods  and  experiences.  The  two-hour  interview  was  videotaped 
and  transcribed.  This  article  is  a  distillation  of  that  interview. 

The  theoretical  basis  for  this  article  is  two-fold.  The  first 
argument  has  two  parts.  It  has  been  argued  that  technology 
precedes  science  in  most  cases  (Basalla,  1988).  Carroll  (1990)  argued 
that  in  fields  like  instructional  design  the  artifact  also  precedes  the 
theory.  Carroll's  argim:ient  about  instructional  design  is  based  upon 
the  work  of  people  like  Basalla  who  have  studied  the  history  of 
technology.  I  will  first  discuss  Basalla's  thesis  and  then  return  to 
Carroll. 

Historians  of  technology  like  Basalla  have  foimd  that  technology 
is  very  rarely  the  product  of  programmatic  scientific  research.  The 
roots  of  technology  characteristically  lie  in  earlier  technologies  or 
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ultimately  in  analogies  to  nature,  and  often  the  moment  of  invention 
is  clouded  in  ambiguity.  Machines  like  the  steam  engine  did  not 
appear  fully-formed  as  the  result  of  deduction  from  scientific 
principles.  Basalla's  argument,  bolstered  by  a  detailed  analysis  of 
not  only  the  steam  engine,  but  also  barbed  wire,  the  cotton  gin,  and 
many  other  interesting  examples,  is  that  technological  evolution  is 
literally  a  kind  of  evolution.  As  such  it  requires  that  four  conditions 
be  present:  diversity,  continuity,  novelty,  and  selection.  Basalla 
shows  that,  in  the  case  of  steam  engines  for  instance,  it  is  very  hard 
to  determine  where  and  when  a  teclmology  was  invented,  because 
all  technology  derives  from  earlier  forms  and  its  survival  and 
development  require  that  numerous  fortuitous  and  local  conditions 
be  present.  For  example,  early  steam  engines  drove  pumps  at  coal 
mines  which  coincidentally  produced  fuel  for  the  engines.  This 
fortuitous  connection  afforded  an  ecologically  friendly  niche  which 
provided  the  continuity  necessary  for  the  steam  engine  to  be 
developed.  Of  course,  theoretical  knowledge  of  the  vacuum  and  its 
practical  applications  probably  contributed  to  the  evolution  of  the 
steam  engine,  but  it  in  no  way  determined  the  application.  Neither 
was  there  a  systematic  program  of  research.  As  a  result,  Basalla's 
position  seems  to  be  that  the  evolution  of  technology  does  not 
require,  and  indeed  historically  has  not  involved,  the  systematic 
application  of  theoretical  principles.  If  technological  evolution  is  like 
natural  evolution,  all  that  is  required  is  a  few  people  with  interesting 
ideas  (novelty)  and  an  atmosphere  in  which  various  ideas  can  be 
tried  out  for  a  while  (diversity),  a  means  of  insuring  continuous 
development  (continuity),  and  a  mechanism  for  choosing  good 
designs  and  eliminating  bad  ones  (selectivity).  In  BasaUa's  terms 
LS's  course  is  a  form  of  technology.  It  has  its  ecological  niche  at  the 
University  of  Kansas.  However,  without  some  formal  record  of  his 
methods,  it  will  not  have  the  opportunity  to  spread  and  survive  in 
the  face  of  a  selective  enviromnent.  This  paper  is  a  way  of 
encouraging  the  development  of  a  particular  technology. 

Carroll's  argument  about  instructional  technology  is  based  upon 
the  thesis  that  science  typically  derives  from  technology.  Thus,  the 
successful  lesson  precedes  and  forms  the  basis  for  a  theory  of 
instructional  design.  Carroll  argued  that  there  are  no  psychological 
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principles  from  which  one  can  rigorously  deduce  instructional 
designs.  The  notion  that  there  are  is  based  upon  the  mythology  that 
tedmology  derives  from  science,  which  Basalla  and  others  have 
exploded.  Indeed,  the  few  psychological  laws  that  exist  (e.g..  Hick's 
Law,  Fitts'  Law)  are  immenaoned  in  instructional  design  texts. 
Carroll  argued  that  instead  of  instructional  design  being  based  on 
psychology,  a  psychological  theory  of  instruction  should  be  based 
upon  actual  instructional  designs,  leading  to  what  he  calls  design- 
based  theory.  Thus,  a  theory  of  instructional  design  should  be  based 
on  interesting  and  successful  instructional  designs.  LS's  course  is  one 
such  design. 

The  second  theoretical  basis  for  this  article  is  related  to  the 
learning  and  teaching  of  complex,  open  skills.  Originally,  the 
distinction  between  closed  and  open  skills  referred  to  skills  learned 
in  predictable  versus  unpredictable  environments  (Poulton,  1957). 
Skills  can  be  classified  in  many  ways  (Colley  &  Beech,  1989)  but  the 
closed-open  distinction  is  most  relevant  here.  For  instructional 
design  purposes,  we  may  think  of  closed  skills  as  those  which  embody 
a  standard  or  "correct"  procedure  and  open  skiUs  as  having  no  one 
''correct"  form.  Since  there  is  no  one  correct  form  the  selection  of 
examples  for  instruction  is  problematic.  I  (Tripp,  1992)  have  argued 
elsewhere  that  complex  professional  open  skills  of  the  type  found  at 
the  top  of  Jones,  Li  and  Merrill's  (1990)  enterprise  hierarchy  are 
traditionaUy  learned  by  being  exposed  to  "masterful"  work.  The  use 
of  traditional  evidence  is  required  by  the  fact  that  little  is  known 
about  the  acquisition  of  complex,  professional  skills.  Beech  &  Colley 
(1989)  reported  that,  '^One  gap  in  our  knowledge  of  the  development 
of  expertise  in  most  fields... is  that  there  is  much  still  to  be  discovered 
about  how  experts  acquire  and  structure  their  knowledge"  p.  328. 
What  is  known  is  that  mastery  of  complex  skills  is  difficult.  Baltes 
and  Smith  (1990)  stated  that  achieving  expertise  requires  at  least, 
"...10  years  and  a  well-designed  program  of  practice,  training,  and 
mentorship"  (p.  99).  The  acquisition  of  skills  is  often  attributed  to 
processes  like  analogy  (Anderson,  1982)  and  imitation  (Annett,  1989). 
For  analogy  or  imitation  to  take  place  the  student  must  have 
internalized  some  kind  representation  of  the  skiU  to  be  learned.  For 
complex  skills  there  is  evidence  that  such  representations  are  like 
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Roschian  prototypes.  For  example  Sternberg  (1990)  treats  wisdom 
as  a  complex  skUl  and  concludes  that  people  carry  and  use  Roschian 
prototypes  of  wisdom  in  their  heads.  Such  prototypes  can  only  have 
come  from  exposure  to  exemplary  cases  of  the  complex  skill  in 
question.  Anecdotal  evidence  tliat  this  is  true  can  be  f  oimd  in  the 
reflections  of  many  skiUed  professionals  who  attribute  their  mastery 
of  a  skill  to  the  influence  of  the  work  of  some  ''master''  of  their  craft. 
Such  masterpieces  consist  of  products  (poems,  buildings,  bridges, 
paintings)  and  performances  (music,  movies,  videotapes,  live 
concerts,  lectures,  etc.).  To  cite  just  one  example,  consider  this 
accoimt  of  the  influence  of  R.  B,  Woodward.  Nobel  Prize  Laureate 
Robert  Biuns  Woodward  was  one  of  the  greatest  chemists  of  this 
century  (Woodward,  1989).  Woodward's  lectures  were  legendary 
and  C.  E.  Woodward  quotes  Barton  as  follows: 

!t  was  a  brilliant  demonstration  of  how  you  could  take  facts  in  the  literature  which 
seemed  obvious  and  just  by  thinking  about  them,  as  he  so  ably  did,  interpret  them  and 
obtain  results  and  then  go  into  the  latx>ratory  and  prove  it  was  the  right  result  That 
we  thought  was  the  work  of  genius....  Ten  years  later  [at  Imperial  College]  our  second 
year  undergraduate  students  could  do  the  problem,  and ...  about  25%  of  them  could  get  it 
right.  Now  does  that  mean  that  in  1958  we  had  25%  mini-Woodwards  in  the  second 
year  class?  No,  of  course  it  didn't.  What  it  meant  was  that  Woodward  had  taught  us 
organic  chemists  how  to  think,  {p,  240) 

Such  education  resembles  the  master-apprentice  relationships 
which  prevailed  in  the  crafts  for  centuries.  It  follows  from  this 
argument  that  instructional  design  should  be  taught  in  part  by 
exposing  students  to  masterful  instructional  designs.  However,  a 
course  taught  by  a  master  teacher  is  largely  unrecorded,  so  there  is 
little  possibility  of  it  influencing  instructional  designers.  Even  if  it 
were  videotaped,  there  is  at  present  no  mechanism  for  distribution  of 
that  recording.  Great  instructional  designs  live  and  die  with  their 
inventors.  Thus,  one  of  Basalla's  criteria  for  technological 
development,  continuity,  cannot  be  satisfied.  This  paper,  while  not 
being  an  actual  recording  of  LS's  teaching,  is  an  attempt  to  ensure 
continuity  by  making  LS's  technology  known  to  instructional 
designers.  By  interviewing  a  professor  with  many  years  of  teaching 
experience,  and  who  has  been  honored  for  the  quality  of  his 
ii\struction,  and  communicating  that  professor's  insights  to  others 
technological  evolution  may  be  encouraged.  Additionally,  because 
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LS  has  incorporated  technology  into  his  teaching,  his  insights  may  be 
especially  interesting  to  the  members  of  this  organization. 


Historical  Background 

Nothing  begins  at  zero.  Almost  30  years  ago  LS  inherited  a 
business  statistics  course  which  was  taught  in  several  sections  of  50- 
60  students  each.  Due  to  some  changes  in  staffing  the  sections  were 
soon  combined  into  one  large  lecture  with  more  than  100  students. 
LS  had  had  no  teacher  training,  but  had  served  as  a  teaching 
assistant  while  doing  graduate  work  at  another  university.  One 
thing  this  experience  taught  him  by  negative  example,  was  that 
public  speaking  skills  were  important  for  a  teacher.  LS  believes  that 
much  of  his  skill  has  come  from  watching  other  people  teach  and 
consciously  copying  their  strengths.  He  happened  to  be  part  of  a 
summer  program  that  involved  teams  of  teachers  who  sat  in  on  each 
others  classes.  One  of  the  teachers  was  very  skilled  in  the  case 
teaching  method.  LS  now  incorporates  cases  into  his  classes  where 
possible.  Most  of  LS's  instructional  design,  however,  seems  to  have 
come  from  years  of  tinkering  and  experimenting  with  different 
techniques  and  from  a  relentless  desire  to  improve. 


Beliefs 

Underl)dng  any  technology  of  teaching  are  the  web  of  beliefs 
that  cause  an  instructor  to  look  at  the  problem  in  certain  ways  and 
prescribe  certain  solutions.  LS  has  some  fundamental  beliefs  about 
teaching.  First  he  believes  that  large  one-section  introductory 
courses  can  be  as  good  as  or  better  than  small  multi-section  coui'ses. 
Second  he  believes  that  carefully  identifying  problems  can  lead  to 
better  outcomes. 

One  aspect  of  identifying  problems  is  specifying  the  decision- 
maker's goals.  As  an  instructor,  LS  believes  that  it  is  not  his  role  to 
''teach/'  or  to  convey  skills  and  knowledge.  Rather,  his 
responsibility  is  to  induce  the  students  to  learn.  By  adopting  this 
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perspective,  four  fundamental  observations  become  apparent  to  LS. 
First,  students  must  be  active  to  learn.  If  we  lived  in  an  ideal  world 
where  students  came  well-prepared  to  class,  in  a  mood  to  engage  in 
active  discussion,  and  armed  with  thoughtful  questions,  lecture 
would  suffice.  LS  believes  we  do  not  live  in  such  a  world. 

The  second  observation  LS  makes  is  that  if  we  wish  to  induce 
our  students  to  learn  then  we  must  know  our  students  well.  We  must 
recognize  that  many  students  would  not  be  taking  our  courses  if  they 
were  not  required.  Many  have  a  poor  attitude.  Unless  a  realistic 
picture  of  the  students  is  drawn,  success  is  unlikely.  A  second  aspect 
of  knowing  our  students,  is  knowing  what  we  want  them  to 
accomplish.  This  means  thinking  in  terms  the  knowledge  and  skills 
we  want  students  to  master,  not  in  terms  of  the  material  we  must 
cover.  LS  thinks  that  for  introductory  courses,  there  are  several 
common  goals  that  we  should  have.  First,  the  course  should  be 
challenging.  Second,  the  course  should  increase  the  students' 
literacy  in  the  subject  matter.  Third,  students  shoiild  develop  an 
appreciation  for  the  importcince  of  the  subject.  Lastly,  the  students 
should  gain  some  understanding  of  why  the  teacher  finds  the  subject 
exciting.  Additionally,  although  the  course  may  be  called  an 
introduction,  it  may  well  be  the  last  course  on  this  subject  students 
will  take.  Because  of  that,  it  is  important  that  the  course  have  a 
well-organized  ending.  In  fact,  LS  believes  that  most  courses  have  a 
short  beginning,  a  long  middle,  and  little  or  no  ending.  Given  the 
type  students  we  often  face  LS  believes  that  much  more  emphasis 
should  be  placed  on  the  beginning  and  ending  of  courses.  It  is  there 
that  most  of  the  rationale  and  therefore  the  motivation  for  the 
subject  is  generated. 

LS's  third  observation  is  that  is  one  thing  to  tell  students  to 
learn  and  it  is  quite  another  thing  to  actually  get  them  to  do  so.  For 
that  reason,  motivating  students  becomes  a  major  part  of  course 
planning. 

The  final  observation  is  that  students  have  individual  learning 
problems  and  therefore  some  method  must  be  devised  to  deal  with 
these  problems  on  an  individual  basis.  LS  believes  that  solving 
problems  in  class  is  a  waste  of  time  for  many  students  because  they 
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could  have  done  it  themselves.  Also,  when  students  have  learning 
difficulties  they  often  do  so  for  individual  reasons,  and  therefore 
answering  their  questions  in  a  recitation  session  is  often  not  useful  to 
other  students.  How  LS  deals  with  these  problems  will  be  discussed 
in  the  next  section. 


The  Course 

LS's  course  is  a  four  credit  class  that  meets  twice  a  week  for 
two  hours  with  a  short  break  between  the  hours.  Typical  enrollment 
is  300  students  impatient  to  get  on  to  the  "real''  business  courses. 
When  students  walk  into  the  400-seat  auditorium  they  see  a  large 
stage  with  three  projector  screens  up  and  running.  The  screen  on  the 
right  may  display  a  hand-vmtten  note  telling  the  students  to  open 
their  books  to  page  247.  The  screen  on  the  left  displays  a  laser- 
printed  transparency  of  the  homework  assignment.  The  screen  in  the 
center  displays  a  computer  generated  "slide-show."  LS  believes  that 
being  set  up  early  and  neatly  communicates  his  seriousness  about  the 
course  and  generates  motivation.  At  exactly  9:30,  he  begins  the  class 
with  a  "Good  Morning"  and  expects  the  students  to  respond.  This 
signals  that  the  class  has  begun  and  also  sets  the  expectation  that 
students  cire  to  respond  to  the  teacher's  words. 

During  the  class,  LS  illustrates  his  lecture  with  computer-based 
slides.  Tliese  have  evolved  from  actual  35-mm  slides  which  he  used 
for  many  years.  The  slides  had  certain  advantages  over  a 
blackboard.  First,  they  are  easier  to  read  in  a  large  auditorium. 
Second,  they  lead  to  better  organization  of  ideas.  Third,  they  allow 
eye-contact  between  the  instructor  and  the  students.  An  instructor 
writing  on  a  blackboard  is  out  of  contact  with  his  students.  There  are 
some  disadvantage  of  such  slides.  First,  they  take  time  to  prepare. 
And  secondly,  they  tend  to  put  too  much  information  up  at  one  time. 
A  teacher  writing  on  a  blackboard  is  naturally  paced-by-ttie  time  it 
takes  to  write  each  word.  For  that  reason,  LS  had  his  slides 
produced  so  that  they  only  reveal  one  line  at  a  time,  and  thus  he  can 
pace  the  amount  of  information  being  revealed.  When  he  changed  to 
computer-based  slides,  he  retained  this  technique.  An  added 
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advantage  of  using  a  computer  is  that  he  can  produce  the  "slides" 
himself  and  also  make  modifications  easily  and  quickly.  He  would 
like  to  move  on  to  the  use  of  animation  and  other  techniques.  He 
doesn't  feel  constrained  by  the  order  of  the  prepared  screens. 
Because  there  are  always  impredictable  questions  LS  keeps  an 
overhead  projector  ready  to  ad  lib  any  displays  he  might  need. 

After  motivation,  the  second  most  important  thing  LS  does,  he 
thinks,  is  designing  the  homework  assignments.  It  is  also  the  most 
time-consuming,  requiring  up  to  an  hour  a  day.  He  does  not  prepare 
a  list  of  assignments  in  advance.  Rather  he  collects  informal 
feedback  from  his  TAs  and  adjusts  each  new  set  of  assigmnents  based 
upon  that  information.  His  TAs  collect  the  assigrunents  before  each 
class  and  correct  them  before  the  next  class. 

LS's  selection  of  problems  is  interesting.  As  we  all  know  the 
questions  at  the  end  of  the  chapter  range  from  easy  to  difficult,  with 
the  first  few  being  easy  and  the  later  ones  being  difficult.  LS  assigns 
the  easy  questions  before  he  lectures  on  the  topic.  This  induces  the 
students  to  read  ahead.  The  difficult  questions  are  assigned  after  the 
topic  has  been  completed,  as  a  kind  of  review.  Thus  each  set  of 
homework  problems  covers  a  range  of  topics  from  what  was  done 
last  week,  to  the  current  topic,  to  what  will  be  covered  next  week. 
The  homework  assignment  also  contains  the  title  of  the  next  lecture, 
so  that  students  see  the  connection  between  the  different  topics.  LS 
does  not  give  his  students  objectives,  except  indirectly  by  pointing  out 
that  certain  topics  are  important. 

LS  makes  his  homework  assigiunents  do  double  duty  by 
presenting  students  early  in  the  course  with  statistical  evidence  that 
those  who  do  wH  on  the  homework  tend  to  do  better  on  the  final 
grade.  This  has  the  effect  of  motivating  the  students,  while 
demonstrating  in  a  relevant  way,  not  only  the  practical  usefulness  of 
statistics,  but  also  the  need  for  presenting  statistical  information 
graphically.  One  semester  he  tried  not  grading  the  homework.  Test 
scores  fell  off  noticeably.  Now  he  uses  those  numbers  to  justify  his 
methods. 
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Testing  is  also  an  important  part  of  LS's  course.  He  gives  three 
intermediate  exams  and  a  final,  all  of  which  are  cumulative  in  their 
coverage.  All  exam  questions  are  public  and  old  exams  are  on  file  in 
the  library.  LS  gives  a  lot  of  exams  because  he  feels  students  do  not 
have  the  discipline  to  stay  current  if  they  are  not  pushed  to  do  so. 
Grading  is  on  a  strict  90, 80, 70-point  basis,  but  the  curve  can  be 
adjusted  by  giving  partial  credit  if  necessary.  LS  has  a  clever  way  of 
motivating  students  to  keep  trying  right  up  to  the  end  of  the  course. 
He  annoimces  that  no  student  will  receive  a  lower  grade  than  his 
grade  on  the  final.  Thus  even  a  student  who  has  done  "<Z"  work 
throughout  the  semester,  still  has  a  chance  of  getting  an  A  or  B  if  he 
does  well  on  the  final. 

LS  believes  that  a  course  shoiild  have  a  structure  like  a  well- 
constructed  staircase.  Tliat  is,  is  should  consist  of  a  series  of  steps  of 
about  equal  height  or  difficulty.  Since  some  topics  are  intrinsically 
hard  than  others,  he  adjust  the  perceived  difficulty  by  adjusting  the 
amount  of  time  spent  on  each  topic.  Another  way  of  adjusting  the 
difficulty  is  by  adjiisting  the  path  by  which  you  reach  a  topic.  Some 
paths  provide  more  preparation  and  some  less. 

LS  uses  teaching  assistants  extensively  to  support  his  course. 
By  arrangement  with  his  dean  he  gets  one  TA  plus  one  TA  for  each  50 
students  enrolled.  Thus  for  300  students  he  gets  seven  TAs.  The  TAs 
are  undergraduates,  chosen  from  his  students  from  the  previous 
semester.  They  are  introduced  at  the  beginning  of  the  semester,  and 
it  is  explained  that  they  are  being  paid  and  have  been  selected  from 
the  best  students  of  the  previous  semester.  Thus,  he  uses  these 
positions  as  motivation  for  students  to  do  well  in  his  class.  The  TAs 
are  half-time  20-hour  per  week  employees.  In  addition  to  grading 
papers  and  tests,  they  attend  every  lecture,  and  they  help  bim  set  up 
his  equipment.  They  also  hold  8-10  office  hours  per  week  when 
students  can  get  help  on  their  homework.  The  offices  are  open  35 
hours  per  week  to  provide  help  and  that  help  is  directed  at  allowing 
the  students  to  complete  their  homework  before  it  is  due.  Since  TAs 
serve  only  for  one  or  two  semesters  they  maintain  enthusiasm  and 
patience  for  what  can  be  a  tedious  job. 
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In  addition  to  the  use  of  the  computer  as  a  screen  display 
device,  LS  has  experimented  with  other  technology.  He  has 
recorded  a  number  of  audio  tapes  which  are  made  available  in  the 
university  language  lab.  These  tapes  covers  topics  which  may  be  a 
prerequisite  to  his  course,  but  which  students  have  failed  to 
understand  completely.  Other  tapes  are  concerned  v^th  problem- 
solving  techniques.  A  third  set  cover  advanced  topics  which  cannot 
be  covered  in  class.  There  are  no  handouts  with  these  tapes  because 
he  want  the  students  to  actively  take  notes.  Another  technique  used 
is  to  pose  questions  and  asking  the  students  to  stop  the  tape  and  think 
about  the  answer. 

Besides  the  tapes,  LS  has  experimented  with  computer- 
assisted  instruction.  These  programs  are  intended  to  illustrate  some 
of  the  major  theoretical  concepts  from  the  course.  The  concepts  are 
presented  in  an  interactive  way  allowing  the  students  to  experiment 
with  various  sets  of  numbers. 


Conclusion 

LS  says  that  he  does  some  of  his  best  teaching  while  he  is 
sleeping.  What  he  means  is  that  by  creating  an  environment  which 
motivates  students  to  study  and  in  which  they  can  get  help,  he  is 
ensuring  that  students  are  learning  while  he  is  home  in  bed.  LS 
claims  he  knows  nothing  about  learning  theory,  but  that  he  is  sure 
that  different  people  learn  differently.  The  trick  is  to  get  them  to  do 
that  learning.  Evidently,  LS  has  discovered  that  trick. 
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Michael  Oakeshott's  Philosophy  of  Education 
and  its  Implications  for  Instructional  Design  Theory  and  Practice 
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University  of  Kansas 

Abstract 

Michael  Oakeshott,  a  political  philosopher,  wrote  extensively  about  education. 
Much  of  what  he  wrote  may  have  interesting  implications  for  instructional  systems 
design.  This  paper  is  an  explication  of  two  aspects  of  Oakeshott's  writings.  The  first 
deals  with  the  nature  of  knowledge  and  its  relationship  to  teaching  and  learning. 
The  second  deals  with  Modem  Rationalism  and  how  Oakeshotf  s  conception  of 
rational  behavior  impinges  on  the  act  of  instructional  design. 


"Learning  is  the  comprehensive  engagement  in  which  we  come  to  know 
ourselves  and  the  world  around  us.  It  is  a  paradoxical  activity:  it  is  doing  and 
submitting  at  the  same  time"  (Fuller,  1989,  p.  43). 

"The  counterpart  of  the  teacher  is  not  the  learner  in  general,  but  the  pupil.  And  I 
am  concerned  with  the  learner  as  pupil,  one  who  learns  from  a  teacher,  one  who 
learns  by  being  taught"  (Fuller,  1989,  p.  44). 

''Education  in  its  most  general  significance  may  be  recognized  as  a  specific 
transaction  which  may  go  on  between  the  generations  of  human  beings  in  which 
newcomers  to  the  scene  are  initiated  into  the  world  they  are  to  inhabit  "(Fuller, 
1989,  p.  63). 


Introduction 

Michael  Oakeshott,  an  English  political  philosopher,  was  greatly  interested  in  the 
question  of  how  people  learn  political  skills.  From  this  he  developed  a  general 
theory  of  knowledge  and  its  acquisition  in  educational  settings.  One  of  his  basic 
assumptions  is  that  there  are  always  important  aspects  of  skill  that  can  never  be 
reduced  to  rules  or  principles.  Because  of  this,  re^  skill  can  only  be  acquired 
through  a  kind  of  apprenticeship  by  being  immersed  in  a  tradition  of  action. 
Additionally,  Oakeshott  is  particularly  critical  of  Modern  Rationalism,  which  he 
believes  commits  the  error  of  believing  that  actions  based  upon  principles  are 
superior  to  actions  based  upon  tradition.  This  paper  attempts  to  explicate  and 
evaluate  these  two  themes  and  then  discusses  their  implications  for  instructional 
design  theory  and  practice. 
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Knowledge 

Michael  Oakeshott  believed  that  all  knowledge  is  constituted  in  various  kinds  of 
abilities.  Abilities  may  be  desaibed  in  tenns  of  propositions  or  rules,  but  the  ability 
always  precedes  the  rules.  Any  prepositional  description  of  an  ability  is  always  an 
abridgment  of  the  ability.  The  ability  always  implies  knowledge  far  beyond  what  can 
be  expressed  in  propositions.  Rules  do  not  exist  prior  to  an  activity,  nor  do  they 
govern  it  or  provide  an  impetus  for  it.  Real  knowledge  only  exists  in  its  use.  When 
an  ability  involves  making  or  doing  and  it  is  significantly  composed  of  physical 
movements  we  call  it  a  skill.  Complex  skills  may  be  composed  of  simpler  skills.  In 
addition,  when  an  ability  is  not  significantly  composed  of  physical  movements  we 
may  still  describe  it  as  skillful. 

However  we  do  not  usually  describe  as  skills  activities  composed  primarily  of 
mental  operations.  Complex  abilities,  such  as  those  associated  with  engineering, 
exploring,  composing,  acting,  teaching,  and  so  forth  would  fall  in  this  category. 
Such  abilities  are  complex  systems  of  simpler  abilities,  grouped  and  aimed  at  a 
specific  focus.  It  is  this  conjunction  of  what  we  know  and  the  way  we  use  it  that 
constitutes  a  crudal  feature  of  Oakeshott's  epistemology.  Knowledge  does  not  exist 
apart  from  its  application.  Indeed,  "What  we  are  aware  of  is  not  a  number  of  items 
of  knowledge  available  for  use,  but  having  powers  of  specific  kinds..."  (Fuller,  1989, 
p.  51).  Abilities  (knowledge)  consist  of  two  components:  information  and 
Judgment.  Information  consists  of  facts,  rules,  concepts  and  principles.  It  is 
impersonal.  Information  can  be  independent  and  inert.  It  only  becomes  useful 
knowledge  to  the  extent  that  it  is  related  to  particular  skills  or  abilities.  An 
important  component  of  information  is  rtiles  and  rule-like  propositions  which  are 
related  to  abilities. 

Every  ability  has  rules.  Rules  are  related  to  abilities  in  one  of  two  ways.  First 
they  may  be  a  prerequisite  to  being  able  to  perform.  For  example  the 
correspondences  between  the  alphabet  and  Morse  Code  must  be  known  before 
messages  may  be  sent  by  telegraph.  Secondly,  rules  may  constitute  a  criterion  for 
determining  the  correctness  of  a  performance,  Oakeshott  says  that  a  grammar  of  a 
language  is  an  example  of  this  kind  of  rule.  It  is  not  required  to  perform  in  the 
language,  nor  is  it  necessary  to  detect  mistakes,  but  it  can  serve  that  purpose.  There 
is  a  third  kind  of  rule-like  proposition  which  may  be  called  a  principle.  Principles 
are  used  to  explain  a  performance;  they  are  never  a  component  of  the  performance 
itself.  Oakeshott  uses  the  example  of  the  mechanics  of  riding  a  bicycle.  The 
principles  of  balance  and  motion  are  usually  completely  unknown  to  the  rider,  nor 
do  they  embody  a  criterion  of  the  performance.  They  belong  to  a  separate  ability,  the 
ability  to  explain  a  performance. 

To  summarize,  all  knowledge  is  embodied  in  abilities,  and  all  abilities  have  an 
informational  component.  However,  this  informational  component  of  knowledge, 
whether  it  be  facts  or  rules,  never  constitutes  all  of  what  we  know.  For  any  concrete 
ability  to  be  realized  it  must  contain  a  component  which  Oakeshott  calls  judgment. 
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Judgment  consists  of  practical  knowledge — ^knowing  how.  It  is  the  tadt 
component  of  knowledge.  Judgment  when  combined  with  information  gives  us 
knowledge  and  allows  us  to  act  and  understand.  Judgment  cannot  be  taught  or 
learned  separately  from  information.  It  is  acqtured  from  the  situation  of  learning 
itself,  through  a  kind  of  apprenticeship.  It  cannot  be  resolved  into  information. 
With  simple  skills  this  judgment  component  may  be  relatively  small,  but  as  we 
ascend  to  higher  levels  of  cognitive  skills,  the  component  of  knowledge  attributable 
to  judgment  is  much  greater.  For  Oakeshott  abilities  do  not  exist  in  the  abstract. 
They  are  to  be  foimd  only  in  concrete  examples.  And  since  each  concrete  example  of 
an  ability  displays  the  style  of  its  performer,  style  is  an  important  aspect  of  any 
ability.  In  a  sense  each  ability  is  like  a  language.  It  is  confined  by  its  grammatical 
rules,  but  each  speaker  will  have  his  own  way  of  expressing  himself.  In  Oakeshotf  s 
opinion  to  ignore  this  aspect  of  knowledge  is  to  miss  "three-quarters  of  the 
meaning"  of  a  performance. 

To  summarize,  for  Oakeshott  knowledge  is  embodied  in  abilities.  Abilities 
consist  of  information  and  judgment.  Information  consists  of  facts  and  rules.  The 
part  of  knowledge  called  judgment  is  tadt  and  implicit.  It  cannot  be  completely 
codified,  but  it  is  the  spring  of  ail  action.  True  knowledge  is  impossible  without  it. 


Learning  and  Teaching 

Education  for  Oakeshott  is  a  specific  kind  of  interaction  between  teachers  and 
learners.  He  is  careful  about  how  he  uses  words  and  especially  stresses  that  he  is 
concerned  not  with  the  learner  in  general,  (because  learning  takes  place  constantly, 
both  formally  and  informally),  but  rather  with  the  learner  as  pupil,  the  learner  as 
counterpart  to  the  teacher.  The  function  of  the  teacher  is  to  initiate  learners  into  the 
inheritance  of  human  achievements.  This  inheritance  is  not  a  world  of  abstraction, 
it  is  a  world  of  beliefs.  These  beliefs  are  expressions  of  the  human  mind  and  form 
an  interlocking  structure  we  call  culture.  The  initiation  of  students  into  this  world 
is  the  job  of  teachers. 

Oakeshott  recognizes  that  learning  may  occur  accidentally  without  the  attention 
of  the  learner  or  the  intention  of  the  teacher.  However,  such  learning  is  not  the 
result  of  teaching.  Teaching  is  a  deliberate  act  designed  to  promote  the  initiation  of 
the  pupil.  It  has  an  order  and  arrangement. 

There  is  an  oft-raised  question  in  educational  cirdes  of  whether  education 
should  be  the  imparting  of  information  or  the  cultivation  of  the  ability  to  learn  and 
think.  Oakeshott  recognizes  this  dilemma  and  tries  to  resolve  it  this  way:  He  sees 
this  question  as  a  difference  in  point  of  view  between  the  pupil  and  the  teacher.  The 
student  naturally  comes  to  education  with  the  desire  to  acquire  information.  He  is 
focused  on  that  which  he  does  not  have.  The  teacher,  seeing  that  information  alone 
cannot  suffice,  seeks  to  impart  a  way  of  tliinking  which  will  allow  that  information 
to  grow  and  prosper.  Thus,  education  is  properly  concerned  with  both  information 


561 

5:1: 

O 

ERIC 


and  learning  to  ieam  but  those  concerns  tend  to  be  emphasized  differently  by 
differing  points  of  view. 

Although  Oakeshott  is  concerned  with  introducing  students  to  their  cultural 
inheritance^  he  does  not  regard  education  as  putting  students  in  touch  with  the 
dead*  Inevitably,  much  of  our  cultural  inheritance  will  be  the  product  of  past 
generations.  However,  the  purpose  of  education  is  not  to  celebrate  the  past,  but  to 
release  the  student  from  ''servitude  to  the  current  dominant  feelings/'  Oakeshott 
recognizes  that  a  teacher  may  be  excused  if  he  finds  no  enthusiasm  for  the  political 
and  material  standards  of  the  current  culture,  but  if  that  person  has  no  confidence  in 
the  past  he  cannot  be  teacher,  for  he  has  nothing  to  teach.  Education,  for  Oakeshott, 
is  an  emancipation  from  the  "tyranny  of  the  momenf'  and  a  step  towards  becoming 
human. 

Education  in  the  inheritance  of  human  achievement  cannot  takes  place  in  the 
abstract  because  that  inheritance  has  no  master  structure.  Our  culture  was  not 
designed,  but  rather  evolved  as  the  result  of  uncountable  individual  actions.  It 
cannot  be  taught  in  principle,  but  must  be  learned  in  its  details.  Also,  teachers  can 
only  present  tiiat  inheritance  in  parts.  It  is  too  large  and  too  contingent  to  allow 
inclusiveness.  Teaching  and  learning,  therefore,  are  always  focused  on  specific 
human  achievements. 

Since  there  are  two  aspects  to  knowledge,  information  and  judgment,  there  must 
be  two  types  of  teaching:  instructing  and  imparting.  Oakeshott  believes  that  the 
teaching  of  information,  called  instruction,  is  best  done  by  direct  commimication. 
Very  often  this  consists  of  communicating  facts  which  have  no  immediate 
application.  Because  children  are  quite  experienced  at  obtaining  information  which 
has  immediate  application,  there  would  be  no  need  for  teachers  if  all  information 
were  of  this  kind.  Since  much  information  is  not  readily  applicable,  we  need 
teachers  to  communicate  this  information  to  students.  The  teacher's  job  is  first  to 
select  and  organize  that  information.  The  teacher  then  has  two  more  tasks.  He 
must  decide  the  order  in  which  the  information  is  to  be  presented  and  then  cause 
the  student  to  practice  with  this  information  so  that  it  may  be  recalled  accurately  and 
readily. 

Education  for  Oakeshott  does  not  end  with  this  practice.  Oakeshott  believes  that 
all  human  activity  involves  knowledge  which  is  beyond  explication  in  rules.  It  is 
this  component,  which  Oakeshott  calls  judgment,  that  must  be  imparted.  It  can 
never  be  communicated  directly.  Also,  it  is  precisely  this  judgment  that  allows 
people  to  do,  make,  understand,  and  explain,  in  other  words,  tliink.  Leanung  to 
think  can  be  taught,  but  not  in  the  absence  of  information.  That  is,  it  is  not  taught 
outside  of  a  lesson  on  arithmetic  or  Latin  grammar.  It  is  a  by-product.  Thinking  is  a 
style  of  doing,  an  idiom,  which  is  revealed  in  the  area  of  action  not  covered  by  rules. 
It  is  learned  by  recognizing  its  effect  in  specific  cases.  Thus  Oakeshott  argues  that  an 
important  part  of  any  learning  situation  is  the  style  of  the  teacher. 
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To  summarize,  Oakeshott  believes  that  education  is  a  deliberate  initiation  of 
pupils  into  their  cultural  heritage.  Since  knowledge  has  both  explicit  and  tadt 
components,  education  consists  of  instructing  and  imparting.  Instructing  is  direct 
and  formal.  It  requires  that  the  teacher  select  what  is  important,  decompose  that 
infonnation,  and  arrange  it  in  an  order  which  respects  its  prerequisites.  The  teacher 
must  then  cause  the  student  to  practice.  Imparting,  on  the  other  hand,  is  indirect 
and  informal.  Teaching  judgment  is  teaching  how  to  think  and  thinking  is  a  way  of 
acting.  For  most  complex  activities,  the  tadt  component  of  knowledge  is  more 
importanl  man  the  explicit  component. 


Rational  Conduct 

The  second  important  aspect  of  Oakeshotf  s  philosophy  of  education  is  his 
interpretation  of  what  it  means  to  say  that  a  person's  behavior  is  rational. 

The  systems  approach  to  instructional  design  is  often  presented  as  a  rational 
approach  to  instructional  problems.  An  instructional  designer  following  the 
systems  approach  is  said  to  be  acting  rationally.  Michael  Oakeshott  (1962)  asked  in 
an  essay  entitled  ''Rational  Conduct"  what  it  means  to  say  that  a  person  is  acting 
rationally  as  opposed  to  urguing  rationally. 

We  have  a  fairly  dear  idea  of  what  a  rational  argument  is  like.  It  is  based  on 
empirical  propositions.  It  uses  logical  patterns  of  reasoning.  It  draws  condusions 
based  upon  acceptable  inferences  and  deductions.  It  avoids  contradictions. 
However,  it  is  not  at  all  clear  what  we  mean  when  we  say  that  someone  is  acting 
rationally.  This  is  an  espedally  interesting  question  for  instructional  design 
theorists  because  instructional  design  is  usually  assumed  to  be  a  rational  form  of 
behavior. 

Oakeshott  points  out  that  the  word,  rational,  has  been  applied  to  many  forms  of 
behavior.  Some  examples  are  rational  agriculture,  rational  diet,  rational  education, 
and  rational  dress.  Professor  Oakeshott  dwells  espedally  on  the  latter  category  and 
examines  a  certain  kind  of  girl's  dothing  called  bloomers  which  were  introduced  in 
the  late  nineteenth  century  (Oakeshott,  1962).  Bloomers  were  a  kind  of  dothing 
considered  to  be  'rational  dress'  for  female  cydists.  They  were  dothes  designed  to 
serve  a  spedfic  purpose;  namely,  to  allow  women  and  girls  to  ride  bicydes 
unencumbered  by  traditional  dress. 

Michael  Oakeshott  asks  why  bloomers  were  thought  to  be  rational.  He  answers 
that  it  seems  to  be  first  because  they  were  adapted  to  drcumstances — ^they  were  a 
successful  solution  to  a  spedfic  problem.  Also,  the  solution  sprang  from  an 
independent  reflective  consideration  of  the  problem.  Thus,  a  'rational  method'  is  a 
form  of  deliberate  conduct  aimed  at  a  purpose  and  governed  only  by  that  purpose. 
Based  on  this  analysis  he  believes  the  following  is  a  fair  account  of  what  people 
often  mean  when  that  talk  of  rational  conduct. 
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•  It  is  a  manner  of  behaving.  It  does  not  depend  on  success. 

•  It  is  behavior  deliberately  directed  to  the  achievement  of  a  formulated  purpose. 

•  It  leads  to  a  simple  end.  If  it  leads  to  a  complex  end  it  must  be  presented  as  a 

series  of  simple  ends. 

•  Rational  conduct  requires  the  availability  of  means  and  a  way  of  selecting 

them. 

•  It  is  not  capricious  or  impulsive.  It  follows  an  explicit  principle. 

•  Its  aim  would  be,  first,  to  determine  a  purpose  to  be  pursued,  secondly,  to 

determine  the  means  to  be  employed  to  achieve  that  (and  no  other)  end,  and 
thirdly  to  act.  (1962,  p.  88) 

This  may  sound  quite  reasonable  to  most  readers  but,  Michael  Oakeshott  believes 
this  account  is  completely  wrong.  Thus  he  writes  that: 

'Tf  this  is  'rational"  behavior,  then  it  is  not  merely  undesirable;  it  is  in  fact 
impossible.  Men  do  not  behave  this  way,  because  they  cannot.  No  doubt 
those  who  have  held  this  theory  have  thought  they  were  describing  a  possible 
form  of  behavior;  and  by  calling  it  'rational',  they  recommended  it  as 
desirable:  but  they  were  under  an  illusion.  No  doubt,  also,  wherever  this 
theory  is  current,  behavior  will  tend  to  conform  to  the  pattern  it  suggests,  but 
it  will  not  succeed.  ...  My  view  is  that  this  is  not  a  satisfactory  notion  of 
rational  conduct  because  it  is  not  a  satisfactory  account  of  any  sort  of  conduct/' 
(l%2,p.89) 

This  conclusion  is  based  in  part  upon  his  previously  described  theory  of 
knowledge.  Knowledge  is  embodied  in  abilities,  and  abilities  always  have  a  tadt 
component  which  cannot  be  explicated.  Thus  behavior  cannot  be  based  upon 
explicit  principles  or  overt  analyses.  Behavior  is  always  the  expression  of 
knowledge  which  is  in  part  implicit. 

Michael  Oakeshott  believes  the  designers  of  the  19th  century  bloomers  did  not 
act  rationally  in  their  sense  of  that  word.  The  clothing  they  designed  v/as  not 
governed  solely  by  mechanical  and  anatomical  considerations.  In  fact,  they  were 
also  governed  by  19th  centiuy  English  feelings  about  what  a  girl's  clothing  should 
look  like.  None  of  this  was  made  explicit  by  the  designers — indeed,  most  of  tliose 
feelings  could  not  be  made  explicit,  because  as  Oakeshott  explains,  action  is  not  a 
manifestation  of  knowledge,  but  rather,  knowledge  is  a  manifestation  of  activity. 
When  a  person  acts,  "...his  actions  are  determined  not  solely  by  his  premeditated 
end,  but  by  what  may  be  called  the  traditions  of  the  activity  to  which  the  project 
belonged"  (1962,  p.  99).  "^Rational'  conduct  is  acting  in  such  a  way  that  the 
coherence  of  the  idiom  of  activity  to  which  the  conduct  belongs  is  preserved  and 
possibly  enhanced"  (1962,  p.  102).  An  idiom  of  activity  is  a  tradition  or  a  knowledge 
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of  hov/  to  behave  appropriately  in  particular  circumstcinces.  The  rules  of  behavior 
of  any  tradition  are  a  mere  abridgment  of  the  activity.  We  know  more  than  we  can 
say.  Traditions  of  activity  are  constantly  changing  things,  changed  by  the  activity 
itself.  "No  action  is  by  itself  'rational'  or  is  'rational'  on  account  of  something  that 
has  gone  before;  what  makes  it  'rational'  is  its  place  in  a  flow  of  sympathy..."  (1962,  p. 


Michael  Oakeshott  finally  ^:<yues  that  he  is  using  the  word  rational  in  the  way  it 
is  normally  used.  He  argues     li  when  courts  of  law  use  the  'reasonable  man' 
standard  (the  standard  that  a  reasonable  man  would  have  acted  thus)  they  are  saying 
that  the  man's  behavior  conforms  to  acceptable  standards.  This  is  a  post  hoc 
judgment;  it  has  nothing  to  do  with  what  he  may  have  thought  before  he  acted. 

The  core  of  the  erroneous  conception  of  behavior  is  a  predisposition  which 
Oakeshott  calls  Modern  Rationalism.  The  characteristics  of  the  Rationalist  are  easy 
to  enumerate.  The  Rationalist  believes  in  the  independent  mind,  free  from  any 
authority  except  reason.  The  Rationalist  believes  that  traditions,  habits,  customs, 
etc,  are  enemies  to  progress.  Progress  for  the  Rationalist  is  the  result  of  disengaged 
reasoning  about  problems.  Indeed,  the  Rationalist  sees  the  world  as  a  series  of 
"problems"  awaiting  rational  solutions.  This  partly  explains  why  Rationalist 
educators  deem  "problem  solving"  to  be  an  important  part  of  education.  The 
Rationalist  never  wants  to  repair  things.  That  would  require  real  knowledge. 
Instead  he  wants  "fimdamental  change/'  The  Rationalist  cannot  see  change  unless 
it  is  self-conscious  change.  Thus,  the  Rationalist  wants  to  replace  tradition  (which 
changes  constantly)  with  principles  (which  by  their  nature  are  inflexible).  Because 
the  Rationalist  derives  his  solutions  from  cold  reasoning  unclouded  by  prejudice, 
he  finds  it  difficult  to  believe  that  anyone  who  thinks  freely  and  honestly  could 
disagree  with  him. 

Oakeshott  believes  that  judgment,  the  tacit  component  of  knowledge  acquired 
though  experience,  is  from  the  Rationalist's  point  of  view  not  knowledge  at  all. 
Thus,  the  Rationalist  emphasizes  the  superiority  of  technical  knowledge,  the 
knowledge  employed  in  overt  reasoning.  Indeed,  for  the  Rationalist  there  is  no 
knowledge  but  technical  knowledge.  He  is  like  a  foreigner  who  sees  nothing  but  the 
surface  characteristics  of  a  society.  For  the  Rationalist,  education  is  nothing  more 
than  training  in  technique,  that  part  of  knowledge  that  can  be  learned  from  books. 

Oakeshott  asks.  Why  is  the  Rationalist  image  attractive?  There  are  several 
reasons.  First  it  serves  as  a  pedagogic  device.  We  tend  to  believe  that  in  order  to 
teach  something  we  must  first  reduce  it  to  set  of  propositions  (the  grammar  of  a 
language,  the  rules  of  research,  the  scientific  method — Oakeshott  does  not  deny  that 
this  may  be  helpful,  only  that  it  is  sufficient).  Another  impulse  to  the  rationalist 
position  is  a  desire  for  mental  honesty.  That  is,  a  disinterested  mind,  free  from 
prejudice,  is  considered  to  be  a  desirable  thing.  Still  another  source  of  support 
comes  from  the  desire  to  begin  from  a  position  of  certainty,  a  position  wluch  only 
independent  propositional  knowledge  seems  to  deliver.  Lastly,  the  call  for  rational 
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conduct  reflects  an  inability  to  deal  with  change.  Rather  than  deal  with  actual 
situations.  Rationalists  seek  to  wipe  the  slate  dean.  Nothing  ever  begins  from  zero 
but  many  reformers  hope  to  do  so.  They  reject  the  present  situation  as  worthless 
and  have  nowhere  to  turn  but  the  "rational"  mind. 

Rationalism  may  be  summarized  as  an  attempt  to  achieve  mistake-proof 
certainty  through  the  use  of  pure  technique,  uncolored  by  prejudice.  K  seeks  to 
substitute  an  overt  method  for  the  covert  judgments  of  tradition.  Rationalism,  for 
Oakeshott,  is  a  relic  of  the  belief  in  magic  in  that  it  seeks  an  unattainable  level  of 
certainty. 


An  Evaluation  of  Oakeshott's  Theories 

In  recounting  Oakeshotf  s  theory  of  knowledge,  constructed  in  the  early  196(ys 
without  reference  to  psychology,  one  may  be  struck  by  how  generally  compatible  it  is 
with  current  psychological  and  computational  theories  of  knowledge.  To  take  one 
prominent  example,  Anderson's  (1983)  theory  of  cognition  posits  two  main  types  of 
knowledge:  declarative  and  procedural.  The  two  types  of  knowledge  resemble 
Oakeshotf  s  information  and  judgment  in  many  ways.  Declarative  representations 
consist  of  cognitive  units.  Units  may  be  propositions,  strings,  or  images,  much  like 
Oakeshotf  s  facts  and  rules.  Declarative  learning  is  abrupt  and  direct,  just  as 
Oakeshott  would  have  it  Procedural  learning,  on  the  other  hand,  is  gradual  and 
inductive,  just  like  the  acquisition  of  judgment.  Procedural  knowledge  consists  of 
productions  (IF-THEN  structures)  which  are  executed  automatically  when  the  IF 
clauses  are  satisfied.  Procedural  knowledge  does  not  use  short-term  memory,  and 
thus  is  tadt.  Learners  are  not  conscious  of  the  contents  of  procedural  memory,  only 
the  capabilities  it  affords.  Thus,  painted  in  broad  strokes  like  this,  there  is  no 
obvious  conflict  between  Oakeshott's  theory  and  this  empirical  psychological  theory. 

Oakeshotf  s  theory  is  not  without  its  problems,  however.  His  insistence  that 
knowledge  is  equivalent  to  ability  seems  counterintuitive.  He  recognizes  that  we 
may  have  information  in  our  heads,  but  that  it  doesn't  become  knowledge  until  it  is 
incorporated  into  an  ability.  Thus,  I  may  "know"  that  a  certain  company  in 
Michigan  sells  epoxy  used  in  wooden  boat  building,  but,  according  to  Oakeshott,  that 
information  is  not  knowledge  if  it  is  not  linked  to  an  ability.  This  is  fine  if  you 
define  knowledge  that  way,  but  if  our  dilemmas  can  be  resolved  simply  by 
redefining  the  words  we  use  then  we  have  no  problems.  Oakeshott  is  using  the 
word  knowledge  in  a  very  unusual  way.  This  unusual  use  of  the  word  knowledge 
is  especially  imfortunate  because  typically  Oakeshott  is  extremely  careful  about 
detecting  the  normal  meaning  of  v/ords  and  using  them  without  distortion. 

In  fact,  Oakeshott  is  ambiguous  about  this  question.  He  states  that  useless 
information  (information  unlinked  to  ability)  is  a  component  of  knowledge.  It  only 
becomes  useful  when  it  attached  to  an  ability.  Thus,  information  is  a  kind  of 
knowledge.  This  ambiguity  is,  in  part,  a  product  of  Oakeshotf  s  style  of  writing.  He 
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deliberately  eschews  the  technical  style  of  modem  academic  writers,  preferring  a 
literary  mode  of  expression.  This  makes  for  pleasant  reading,  but  it  does  not  help 
resolve  certain  crucial  questions  such  as  tine  one  raised  above. 

Additionally,  Oakeshotf  s  insistence  that  all  activities  must  find  their  spring  in 
judgment  is  problematical.  Anderson  (1982),  for  example,  attributes  skill  acquisition 
to  weak  problem-solving  methods  such  as  analogy.  Anderson  believes  that 
declarative  representations  of  examples  are  used  to  construct  action  sequences 
which  allow  approximation  of  skilled  behavior.  If  the  source  of  this  semi-skilled 
activity  is  taken  to  be  the  declarative  representation  then  Oakeshott's  theory  is 
wrong.  Certain  types  of  semi-skilled  or  novice  activity  may  not  have  a  tadt  origin. 

If  Oakeshott  countered  that  the  source  of  the  activity  is  not  the  declarative 
representation,  but  the  use  of  analogy  as  a  problem-solving  strategy,  then  the 
question  is  complicated.  Human  use  of  analogy  is  notoriously  hard  to  explain 
(Vosniadou  &  Ortony,  1989).  Any  two  situations  may  be  similar  in  any  number  of 
arbitrary  ways.  The  argument  might  be  made  that  the  use  of  analogy  is  a  form  of 
tacit  knowledge  and  therefore  the  activity  is  still  rooted  in  judgment.  However,  in 
these  situations  there  seem  to  be  two  activities  going  on  simultaneously.  First,  the 
actor  is  analogizing  in  order  to  find  a  method.  But  secondly,  the  actor  is  executing 
an  activity.  If  this  second  activity  is  thought  of  as  differing  from  the  first,  then 
plainly  it  is  an  activity  that  derives  from  information  not  judgment.  If  this  is  true 
then  Oakeshotf  s  claim  that  all  activity  is  rooted  in  judgment  is  too  strong. 
Oakeshott  position  would  still  hold  for  skilled  activity,  but  not  for  unskilled  activity. 
Perhaps  he  is  referring  only  to  skilled  behavior,  but  that  is  not  the  impression  ones 
gets. 

Another  problem  is  that  Anderson  (1983)  sees  all  procedural  knowledge  as  first 
passing  through  declarative  memory  and  then  transformed  into  productions  by  the 
mechanism  of  practice.  It  is  not  dear  how  judgment  enters  the  mind  in  Oakeshotf  s 
system,  but  it  appears  not  to  pass  though  dedarative  memory.  Again  he  is 
ambiguous  on  tiiis  point.  He  does  assert  that  certain  aspects  of  judgment,  i.e., 
honesty,  patience,  exactness,  industry,  etc.,  are  learned  by  recognizing  them  in  the 
actions  of  a  teacher.  Thus  judgments  seem  to  be  consciously  acquired.  Perhaps  they 
are  like  the  particulars  which  serve  a  general  purpose  whidi  Sdion  (1988)  has 
written  about.  However,  Oakeshott  seems  to  reserve  the  term  judgment  for  only 
those  aspects  of  knowledge  which  are  completely  tadt.  Anderson,  on  the  other 
hand,  would  locate  such  low-level,  and  plainly  overt,  skills  as  typing  in  procedural 
memory,  so  there  is  apparently  some  conflict  between  these  positions.  Oakeshott 
allows  that  low-level  skills  have  a  very  minor  tadt  wmponent,  and  in  the  case  the 
case  of  skills  like  typing,  that  tadt  component  may  be  so  minor  that  it  can  be  safely 
ignored  from  an  educational  point  of  view. 

Leaving  these  problems  aside,  it  is  interesting  to  compare  Oakeshotf  s  theory 
with  certain  knowledge  representation  theories  becoming  well-known  to 
instructional  systems  people.  Jones,  Li,  and  Merrill's  (1990)  computational  system  of 
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knowledge  representation  contains  elements  that  parallel  Oakeshotf  s  theory.  In 
pardcular,  Jones  et  al/s  use  of  enterprises  as  a  central  unit  of  knowledge  seems  to 
resemble  Oakeshotfs  use  of  the  word,  activities.  An  enterprise  is  a  complex 
performance  that  requires  an  integrated  set  of  knowledge  and  skills.  This  sounds 
like  Oakeshott^s  decomposition  of  knowledge  into  information  and  judgment. 
Jones  et  al*,  however,  specifically  decompose  knowledge  into  three  basic 
components:  entities,  activities  and  processes.  Entities  are  things:  objects,  creatures, 
places  or  symbols.  Activities  are  actions  performed  by  the  student.  Processes  are 
actions  or  relations  outside  the  learner.  These  three  components  may  be  linked  by  a 
set  of  defined  relationships.  In  general,  because  Jones  et  al.'s  purpose  is  to  create  an 
entirely  explicit  representation  of  knowledge  they  do  not  consider  that  knowledge 
may  have  a  tadt  component.  Thus,  activities  and  processes  in  Jones  et  al.'$  system 
correspond  to  a  form  of  information  in  Oakeshotf  s  system.  To  the  extent  that  I 
understand  the  Jones  et  al.  system  there  is  no  provision  for  what  Oakeshott  calls 
judgment.  This  is  within  the  instructional  design  tradition  of  treating  knowledge 
and  skills  as  things  that  can  be  made  explicit  and  commimicated  directly.  Thus,  the 
component  of  knowledge  called  judgment  by  Oakeshott  is  not  addressed  by  Jones  et 
al.  As  with  Anderson's  psychological  theory,  Oakeshotf  s  insistence  on  a  central  and 
exclusive  role  for  tacit  knowledge  distinguishes  it  from  other  theories  of  knowledge. 

To  summarize,  Oakeshotf  s  characterization  of  knowledge  has  many  superficial 
communalities  with  current  psychological  and  computational  theories  of 
knowledge.  However,  his  insistence  on  a  central  role  for  judgment,  a  kind  of  tadt 
and  traditional  knowledge,  distinguishes  it  from  other  theories.  In  general,  these 
theories  take  no  account  of  this  kind  of  knowledge. 

Turning  to  Oakeshotf  s  criticism  of  Modern  Rationalism,  it  is  dear  that  he  differs 
radically  with  the  standard  academic  conceptions  of  skill,  expertise,  and  professional 
behavior.  Oakeshotf  s  critidsm  of  Modern  Rationalism  rests  upon  his  theory  of 
knowledge.  If  you  accept  his  assertion  that  activities  are  grounded  in  experience- 
based  judgment,  then  you  will  accept  that  rational  behavior  cannot  be  based  upon 
explidt  prindples.  Since  recently  there  has  been  put  forth  a  hypothesis,  roughly 
compatible  with  Oakeshott,  that  learning  requires  "cognitive  apprenticeships''  and 
that  "all  learning  is  like  language  learning"  (Brown,  Collins,  &  Duguid,  1989),  I  will 
try  to  evaluate  Oakeshotf  s  position  by  reference  to  foreign  language  learning. 

It  is  a  commonplace  that  the  fluent  speaking  of  a  foreign  language  requires  that 
you  automatize  the  rules  and  speak  without  "thinking."  It  is  also  accepted  that 
languages  are  vast  and  that  no  complete  grammar  is  ever  likely  to  be  written.  Thus, 
a  learner  of  Japanese,  for  example,  can  never  leam  the  language  from  books  and 
even  if  you  could  leam  the  rules  from  books  you  would  still  not  be  able  to  speak 
properly.  Skilled  speaking  of  Japanese  requires  that  you  speak  "like  a  Japanese," 
plainly  a  vague,  though  concrete,  aiterion.  Oakeshott  would,  of  course,  hold  that 
skilled  behavior,  like  speaking  a  foreign  language,  contains  a  large  tacit  component, 
and  that  the  rules  of  such  an  activity  can  never  be  made  completely  explicit.  Thus, 
for  language  learning  at  least,  Oakeshotf  s  predictions  seem  to  hold  and  if  all 
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learning  is  indeed  like  language  learning  as  Brown  et  al.  hold  then  cognitive 
apprenticeships,  if  they  include  immersion  in  the  traditions  of  action,  wotdd  be 
exactly  what  Oakeshott  would  recommend. 

His  argument,  of  course,  is  that  all  forms  of  skilled  behavior  are  exactly  like 
language  learning.  Skilled  behavior  requires  that  you  behave  like  a  native,  whether 
you  reside  in  a  foreign  countiy  or  the  country  of  scientists  or  plumbers.  Rational 
behavior,  for  Oakeshott,  is  precisely  this  kind  of  "native"  intuition  about 
appropriate  behavior.  Since  speaking  a  foreign  language  according  to  explicit  rules 
would  result  in  a  less-skilled  form  of  behavior  it  follows  that  any  activity  based  on  a 
set  of  principles  will  also  be  inferior  to  action  based  on  the  traditions  of  the  activity. 
For  Oakeshott,  the  only  way  to  acquire  this  authentic  kind  of  rationality  is  to  be 
immersed  in  the  culture,  to  have  a  cognitive  apprenticeship. 

Many  readers  will,  no  doubt,  still  find  Oakeshotf  s  position  strange,  even  weird. 
They  are  so  accustomed  to  (they  were  raised  in  a  tradition  oO  seeing  action  based 
upon  explicit  principles  and  methods  as  the  preferred  representation  of  professional 
behavior  that  they  will  find  this  theory  unacceptable.  Their  conceptiors  of 
instructional  design  are  based  upon  a  rationalist  theory  of  knowledge  and  action. 
Standard  prescriptions  for  instructional  design  fairly  replicate  Oakeshotf  s 
characterization  of  Rationalism.  We  typically  talk  about  clarifying  purposes, 
determining  objectives,  selecting  means,  and  acting  based  upon  our  analysis.  It  is 
important  to  point  out,  therefore,  what  Oakeshott  is  not  saying.  He  is  not  saying 
that,  in  the  absence  of  tru;^  and  rich  experience,  intuition  is  superior  to  explicit 
methodology.  What  he  is  saying  is  that  true  skill,  in  addition  to  requiring  explicit 
informational  knowledge,  also  requires  massive  exposure  to  the  traditions  of  a 
domain  of  activity.  Also,  he  is  not  saying  that  behavior  based  upon  explicit 
principles  is  doomed  to  failure.  It  will  often  succeed,  not  because  of  the  principles, 
but  because,  at  crucial  moments,  the  actors  will  abandon  principles  and  act  according 
to  their  intmtive  judgments  based  upon  whatever  experiences  they  may  have. 
Since  tacit  judgments  occur  nearly  transparently  it  is  very  easy  to  ignore  or  devalue 
them.  One  need  only  set  foot  in  a  foreign  country  to  appreciate  that  most  of  the 
actions  we  perform  unthinkingly  every  day  do  not  ''make  sense"  to  other  people. 
(Question:  How  many  times  do  you  knock  on  a  door  in  the  USA?    What  does  it 
mean  if  you  vary  the  number  of  knocks?) 

Precisely  because  judgment  is  transparent,  to  most  instructional  designers  it  is 
not  considered  important.  However,  it  has  not  gone  completely  unnoticed.  Schon 
(1983,  1987)  has  recorded  the  problems  of  architecture  students  who  complain  that 
most  of  the  decisions  made  by  their  teachers  are  covert.  The  traditional  solution  to 
this  problem  in  architecture  has  been  to  require  a  studio  experience  of  all  students. 
This  is  interesting  because  architecture,  as  a  discipline,  evolved  before  the  age  of 
Modem  Rationalism  and  the  studio  experience  is  precisely  the  kind  of 
apprenticeship  Oakeshott  would  prescribe.  If  these  examples  of  language  learning 
and  architecture  are  representative  of  skilled  behavior  in  general,  it  appears  that 
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Oakeshott  has  made  a  well-founded  criticism  of  the  Rationalist  account  of  the 
proper  basis  of  behavion 

In  siun^  although  Oakeshotf  s  conception  of  rationality  and  his  criticism  of 
Rationalism  may  appear  strange  at  first,  it  follows  directly  from  his  theory  of 
knowledge.  If  correct,  he  has  made  a  major  contribution  to  knowledge  by  pointing 
out  that  true  skiUed  behavior  is  acquired  by  initiation  in  a  tradition,  not  just  by 
studying  technical  knowledge  encoded  in  principles.  Essentially  what  he  is  saying  is 
that  tradition  is  a  special  form  of  technology.  It  is  a  way  of  acting  and  thinking  that 
captures  important  tadt  knowledge  which  can  never  be  made  completely  explicit. 
As  such  it  is  an  essential  component  of  intelligent  action. 


Implications  for  Instructional  Systems  Design 


ISD  methodology 

On  one  level  at  least  Oakeshotf  s  ideas  seem  very  compatible  with  typical  ISD 
methods.  His  conception  of  knowledge  as  ability  suggests  that  knowledge  can  only 
exist  in  actions.  Since  ISD  people  are  accustomed  to  thinking  that  lessons  are  focused 
around  behavioral  objectives,  there  is  an  obvious  affinity  between  the  two  positions. 
If  knowledge  is  a  manifestation  of  activity,  it  makes  sense  to  measure  knowledge  by 
observable  actions.  Additionally,  Oakeshott  believes  that  complex  skills  are 
conglomerations  of  subskills  that  can  be  separated  out  and  approached  individually. 
Skills  also  have  hierarchical  relationships  with  certain  facts  and  rules  which  are 
prerequisites  to  mastery.  All  of  this  seems  familiar  and  uncontroversial, 
Oakeshotf  s  conception  of  the  teacher  as  a  person  who  selects  content,  orders  it, 
presents  it,  and  causes  the  student  to  practice  with  it  is  unexceptional.  Even 
Oakeshotf  s  disinterest  in  discovery  learning,  is  not  a  problem  for  ISD  people^  who 
generally  prefer  direct  instruction. 

However,  when  Oakeshott  starts  talking  about  imparting  judgment  to  students, 
he  parts  company  with  ISD.  Normal  ISD  has  no  prescriptions  for  such  knowledge 
because  it  is  not  recognized  as  knowledge  at  all. 

This  need  not  be  the  case  though.  One  way  of  thinking  about  the  imparting  of 
jur'^gments  is  to  use  a  skills  matrix,  Peters  (196J3)  criticized  Oakeshott  by  asserting 
that  to  the  extent  that  a  skill  is  directed  toward  a  well-defined  goal  it  need  not  be 
learned  by  apprenticeship  and  could  be  learned  by  self-study.  Another  way  of 
expressing  this  idea  is  to  make  a  distinction  between  open  and  closed  skills. 

Originally,  dosed/open  skills  referred  to  those  learned  in  a  predictable  or 
unpredictable  environment  (Poulton,  1957),  For  instructional  design  purposes, 
dosed  skills  may  be  thought  of  those  which  embody  a  standard  or  "correct" 


5'I;y570 


13 


prcKedure;  open  skills  have  no  one  "correct"  form.  For  example,  the  setting  of  the 
time  on  a  digital  watch,  a  dosed  skill,  has  a  defined  procedure  which,  if  followed, 
usually  guarantees  success.  On  the  other  hand,  a  game  such  as  chess,  although 
having  well-defined  components,  has  no  set  of  moves  which  v/ill  guarantee  success. 
Chess-playing,  like  most  game-like  activities,  is  an  open  skill.  (Open  skills  like  chess 
playing  often  have  closed  skill  subcomponents.) 


Discrete: 

Continuous: 

Qosed: 

•  operating  many  push-button  devices. 

•  math  operations 

•  typing. 

•  many  aspects  of  written  language. 

•  Speaking  sentences  in  foreign 
language. 

•  repairing  simple  things 

•  making  simple  objects — not  works  of 
art  (carpentry,  dressmaking,  cooking) 

•  reading  &  writing  (basic  skills) 

•  basic  computer  operations 

•  controlling  many  devices  that  involve 
motion,  (dentist  drill,  lathe,  airplane, 
sailboat,  bicycle) 

•  taking  dictation. 

•  refiereeing  a  game  (basic  level) 

•  basic  listening  comprehension 
(foreign  language) 

«  swimming  (and  most  non-competitive 
sports) 

•  auctioneering 

•  sui^gery 

•  wdding 

Open: 

•  writing  an  essay 

•  painting  (art) 

•  planning  a  lesson 
®  games  (chess) 

•  architecture 

•  most  kinds  of  design. 

•  skilled  cooking 

•  writing  a  proposal 

•  skilled  computer  programming 

•  photography 

•  acting 

•  singing 

•  teaching 

•  conversing  in  a  foreign  language 

•  competitive  sports 

•  negotiation/debate 

•  lawyer  interrogating  a  witness 

•  managing  a  project 

•  leadership 

•  warfare 

Rgurel.  Skill  Matrix:  Examples 


In  order  to  construct  a  matrix  it  is  necessary  to  introduce,  in  addition  to  the 
closed  and  open  dimension,  another  dimension  based  on  discrete  and  continuous 
skills.  Discrete  skills  traditionally  are  skills  under  the  control  of  a  discrete  stimulus. 
However,  for  instructional  design  purposes,  they  may  be  thought  of  as  skills  which 
are  under  no  time  constraints.  By  no  time  constraints,  I  mean  that  the  procedure 
may  be  performed  slowly  with  interruptions  and  still  achieve  success.  An  example 
of  a  discrete  skill  is  programming  a  VCR.  One  can  successfully  program  the  VCR 
from  a  manual  without  practice.  Of  course  many  discrete  skills,  with  practice, 
become  smooth  and  automatic  and  may  appear  to  be  continuous.  Typing  is  an 
example.  However,  one  may  stop  or  slow  down  at  any  time  and  still  achieve  a 
successful  product. 
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In  contrast,  continuous  skills,  traditionally  defined  as  skills  under  the  control  of 
a  continuous  stimulus,  may  be  thought  of  instructionally  as  skills  performed  under 
the  constraint  of  real-time.  The  time  constraint  is  a  result  of  the  fact  that  these  skills 
involve  reacting  to  a  continuously  changing  situation  which  is  at  least  partly  out  of 
the  control  of  the  actor.  Many  of  these  skills  involve  continuous  motion,  such  as 
swimming  or  dancing.  The  pace  of  the  action  is  dictated  by  nature  (gravity)  in  the 
case  of  swimming,  whereas  for  dance,  rhythm  and  the  presence  of  other  dancers 
form  a  constraint.  In  addition  to  such  physical  skills,  many  business  skills,  like 
negotiating  or  interviewing,  are  also  continuous.  The  key  factor  in  many 
continuous  skills  is  that  the  actor  is  reacting  to  a  changing  environment  which  is 
not  completely  under  his/her  control.  By  combining  the  two  contrasting 
classifications,  one  can  produce  a  2x2  matrix  of  skills  (see  Figure  1).  Note  that  the 
examples  given  may  be  ambiguous.  For  example  cooking  v/hich  is  called  discrete, 
include  continuous  skills,  like  frying.  In  many  cases,  the  distinction  between  closed 
and  open  is  often  a  distinction  between  normal,  basic  skills  and  high  mastery.  High 
mastery  of  a  skill  is  almost  always  open. 

Viewed  from  the  point  of  view  it  appears  that  Oakeshott  probably  had  open  skills 
in  mind  when  he  constructed  his  theory.  It  is  very  improbable  that  programming  a 
VCR  involves  any  "style''  or  judgment,  in  his  sense.  Jii  general,  if  the  student 
masters  the  correct  procedure,  success  is  achieved.  Originality  is  not  desired. 

One  important  difference  between  open  and  closed  skills  is  the  kinds  of  examples 
that  the  students  may  be  exposed  to.  Since  closed  skills  have  a  correct  form  the 
instructor  will  expose  the  students  to  that  form.  Students  can  practice  by  mimicking 
the  correct  performance  and,  very  often,  simple  right-or-wrong  feedback  will  suffice 
because  the  student  can  check  his/her  performance  against  the  correct  model. 

On  the  other  hand,  open  skills,  because  they  have  no  one  correct  form  by 
definition,  present  an  instructional  design  problem.  It  is  here  that  Oakeshotf  s  ideas 
are  helpful.  It  appears  that  real  mastery  of  open  skills,  in  general,  does  reqtihe  the 
kind  of  judgment  that  can  never  be  reduced  to  rules.  One  way  of  analyzing  this 
problem  is  to  consider  the  kind  of  teaching  used  in  traditionad  open-skills  education. 
By  traditional  open  skills  I  mean  architecture,  art,  literature,  music,  theater  and  the 
like. 

In  general  traditional  education  in  open  skills  has  involved  exposing  the  student 
to  ''masterworks.''  It  is  useful  to  note  that  discrete  open  skills  generally  result  in  a 
physical  product,  whereas  continuous  open  skills  result  in  a  performance.  Thus  a 
masterwork  can  include  products  such  as  buildings,  books,  and  paintings  as  well  as 
recordings  of  dances,  plays,  and  music  Using  this  kind  of  empirical  data,  we  can 
conclude  that  open-skills  education,  if  it  is  to  recognize  Oakeshotf  s  judgment 
component  of  knowledge,  must  include  an  apprenticeship  which  exposes  students 
to  masterworks.  This  suggests  that  the  teacher,  although  important,  is  not  the  only 
model  for  judgment.  Peters  (1968)  noted  this  probl*=»m  by  referring  to  the  work  of 
coaches  who  elicit  skilled  performance  from  athletes  without  being  "masters'' 
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themselves.  Of  course,  student-athletes  are  well  aware  of  the  performances  of  great 
athletes,  and  these  performances  probably  serve  the  same  purpose  for  the  athletes  as 
Shakespeare  for  literature  students  or  Picasso  for  art  students. 

Although  file  prescription  of  using  masterworks  to  teach  open  skills  is 
straightforward,  its  implementation  is  problematical.  In  general,  the  traditional 
open  skills  have,  over  many  generations,  developed  a  consensus  concerning  what 
actually  con  >titutes  a  masterwork,  and  have  generated  ''museums,"  both  literally 
and  figuratively,  of  examples  of  masterworks.  Even  chess  players  have  published 
collections  of  grec;.*  games  by  master  champions  of  the  past  which  they  can  recreate 
and  study.  For  many  of  the  open  skills  which  instructional  designers  are  trying  to 
teach,  there  exists  not  such  consensus.  Thus,  there  are  no  "museums"  of  great 
negotiators,  managers,  or  even  teachers.  If  Oakeshotf  s  ideas  aie  accepted,  the 
creation  of  such  museums  will  be  a  priority.   It  is  here  that  technical  means  such  as 
the  videodisk  or  multimedia  can  be  helpful.  Since  ISD  has  a  long  association  with 
technology,  this  will  interface  well  with  current  practice. 

In  spite  of  the  tmconventional  appearance  of  Oakeshotf  s  theory,  from  a 
traditional  ISD  viewpoint,  I  have  tried  to  show  that  his  ideas  can  be  incorporated 
into  standard  ISD  practice.  By  viewing  judgment  as  a  component  of  open  skills,  and 
taking  traditional  open  skills  education  as  an  empirical  basis,  fairly  simple  and 
straightforward  implications  for  ISD  can  be  drawn.  Implementation  will  require  the 
construction  of  a  collection  of  masterworks  but  this  is  well  within  the  capability  of 
instructional  technology. 


The  Education  of  Instructional  Designers. 

The  systems  approach  to  instructional  design  has  long  been  considered  rational 
because  it  sets  goals  and  pursues  them  based  upon  empirical  principles,  free  from 
folklore  and  superstition.  However  if  Oakeshotf  s  definition  of  rationality  is 
accepted,  ISD  must  be  considered  rational  only  if  the  instructional  systems  designer 
is  acting  within  a  tradition  of  activity  known  as  instructional  systems  design.  TTiis 
raises  questions  about  the  proper  education  of  instructional  designers. 

If  is  fair  to  say  that  current  education  consists  of  introducing  students  to 
instructional  design  principles  and  then  causing  the  students  to  practice  in  semi- 
realistic  and  realistic  situations.  Many  students  also  experience  internships. 
Students  are,  to  my  knowledge,  not  deliberately  exposed  to  the  masterworks  of  ISD. 

One  way  of  thinking  about  how  the  education  of  ISD  students  could  be  different 
is  to  examine  the  education  of  architects.  The  education  of  architects,  a  traditional 
profession,  evolved  before  modem  times.  Students  naturally  are  exposed  to  models 
or  pictures  of  the  great  buildings  of  the  past.  This  part  of  their  education  occurs 
almost  unconsciously.  Another  essential  part  of  their  education  is  the  design 
studio. 
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The  design  studio  experience  has  traditionally  been  a  component  of  architectural 
education.  The  idea  that  the  architectural  studio  could  serve  as  a  model  for 
professional  education  has  been  well  developed  by  Schon  in  his  book.  Educating  the 
Reflective  Practitioner.  (The  following  account  is  based  upon  conversations  with 
architecture  students  and  professors,)  The  architectural  design  studio,  as  a 
pedagogical  method,  has  no  parallel  in  its  intensity  and  involvement,  save  perhaps 
the  internship  of  medical  students.  The  studio  experience  consists  of  several  key 
components.  Among  these  are  a  problematic  situation,  a  studio  teacher,  a  jury  of 
experts ,  and  a  place,  called  a  studio,  dedicated  to  the  design  activities  of  the  course. 
The  problematic  situation  is  intended  to  represent  the  real  world  of  practice,  but  be 
relatively  free  of  its  pressures,  distractions,  and  risks.  The  studio  teacher  guides  the 
students  through  their  design  projects,  while  sharing  his  knowledge  and  experience. 
He  functions  less  as  a  teacher  than  as  a  coach  who  demonstrates,  advises,  criticizes 
and  questions.  The  jury  of  experts,  recruited  from  imiversity  faculty,  serves  two 
roles.  The  first  is  to  participate  in  the  studio,  sharing  their  suggestions  and 
criticisms  with  the  students.  In  some  ways  this  resembles  a  master-apprentice 
relationship,  and  it  allows  the  students  to  see  how  an  experienced  professor  would 
deal  with  real  world  problems — something  they  almost  never  encounter  in 
conventional  courses.  The  second  role  of  the  jury  is  to  judge  the  students'  projects 
at  the  end  of  the  semester.  Projects  are  displayed  and  comments  are  given  publicly. 
The  jury's  judgment  in  many  cases  is  the  grade  for  the  project. 

The  final  component  of  the  studio  experience  is  the  studio  itself,  A  studio  is  a 
dedicated  workplace.  In  architectural  sdiools,  when  students  enroll  in  a  studio 
course  they  are  typically  assigned  a  desk  and  equipment  which  "belong"  to  them  for 
the  duration.  Since  the  workplace  is  restricted  to  the  enrolled  students  and  their 
teachers  there  is  a  highly  sodal  component  in  the  studio  experience.  Students 
cooperate  with  each  other  and  learn  from  watching  other  projects  take  form.  There 
is  also  an  element  of  competition  which  arises  from  the  fact  that  all  are  striving  to 
solve  the  same  problem. 

As  a  result  of  the  studio  experience,  students  can  be  expected  to  have  gained  at 
least  three  types  of  knowledge.  The  first  involves  a  new  ability  to  represent 
problems  and  synthesize  solutions.  The  second  involves  creative  approaches  to 
problem-discovery  and  problem-solving.  The  third  involves  a  new  critical  language 
which  allows  them  to  discuss  problems  and  judge  solutions.  Although  judgment 
(in  Oakeshotf  s  sense)  is  involved  in  all  three  types  of  knowledge,  it  is  especially  in 
learning  the  critical  language  that  students  become  initiated  into  the  traditions  of 
their  profession.  The  educational  experience  in  the  design  studio  involves  all  three 
of  these  tyi>es  of  learning,  but  it  does  so  simultaneously.  Learning  takes  place 
primarily  through  example  and  experience  rather  than  through  explanation.  In  this 
way  the  studio  is  intended  to  provide  a  bridge  between  theory  and  practice. 

Could  this  be  a  model  for  the  education  of  instructional  designers?  It  is  apparent 
that  there  are  many  similaiities  between  architecture  and  ISD,  Both  are  design 
activities.  Both  can  trace  their  roots  to  activities  that  emerged  in  ancient  times. 
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Both  require  techtiical  knowledge*  Both  seek  to  fashion  specific  artifacts  to  fit 
specific  situations.  The  fact  that  architects  design  artifacts  that  exist  in  space  and 
instructional  designers  design  artifacts  (lessons)  that  exist  mostly  in  time,  means 
that  ISD  is  somewhat  more  abstract  and  therefore  difficult  to  acquire.  In  spite  of  this 
difference,  it  seems  apparent  that  a  design  studio  experience  could  be  easily 
incorporated  into  ISD  training.   A  prerequisite  to  such  an  experience  would  be 
exposure  to  the  masterworks  of  ISD*  As  with  other  recent  open  skills,  there  exists 
no  professional  consensus  on  what  those  works  would  be*  There  appears  to  be  no 
theoretical  barrier  to  the  construction  of  such  a  consensus  if  professors  of  ISD  sought 
to  do  so* 

As  for  the  implementation  of  a  design  studio  experience,  there  again  appears  to 
be  no  theoretical  barrier.  Many  practical  obstacles  will  have  to  overcome,  though. 
There  is  no  physical  location  currently  allocated  for  design  studio  experiences.  Space 
is  more  precious  than  gold  at  most  uxiiversities.  There  is  no  tradition  of  studio 
pedagogy.  Perhaps  some  skill  could  be  acquired  if  ISD  professors  audited  (or 
participated  in)  architecture  studio  classes,  but  this  kind  of  skill  will  evolve  slowly. 
The  students  will  not  know  how  to  act.  Architecture  students  have  the  benefit  of 
the  experiences  of  generations  of  students  which  have  set  parameters  for  what  is 
rational  and  irrational  behavior  in  a  studio  class.  Juries  may  be  difficult  to  assemble. 
Schools  of  Education  have  no  provision  for  compensation  for  jury  duty.  There  is 
no  tradition  of  participation.   Jury  members  will  not  know  how  to  behave. 
A  graduate  student  of  mine,  who  was  previously  a  student  of  architecture,  once 
asked  me  why  we  in  education  are  so  kind  when  we  criticize  students'  work.  "In 
architecture,"  she  said,  "if  we  don't  like  something,  we  say  it 's  ugly!"   Our  behavior 
was  irrational  to  her  because  she  had  not  been  sufficiently  initiated  into  our 
traditions.  Instructional  design  juries  will  need  to  develop  a  tradition  of  criticism 
which  meshes  with  the  culture  of  schools  of  education*  The  obstacles  are  many  but 
none  are  insuperable,  if  there  is  a  will  to  take  the  implications  of  Oakeshott's 
philosophy  seriously. 


Conclusions 

Michael  Oakeshotf  s  philosophy  of  education  is  based  upon  a  conception  of  the 
nature  of  knowledge,  its  acquisition  and  use.  Because  he  posits  a  component  of 
knowledge,  judgment,  which  can  never  be  reduced  to  rules,  and  can  only  be  learned 
through  a  kind  of  cognitive  apprenticeship,  his  philosophy  has  implications  for  ISD. 
Those  implications  include  prescriptions  for  the  teaching  of  open-skilled  behavior 
and  the  training  of  instructional  designers.  The  purpose  of  this  paper  has  been  to 
elaborate  that  philosophy  and  its  implications. 
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Inter-High  School  Writing  Project 

Darlene  Chirolas,  University  of  Illinois 

A  project  was  undertaken  by  staff  at  ihe  Computer-based  Education  Research  Laboratory  of  the 
University  of  Illinois  with  two  Illinois  high  Schools,  (one  in  Rantoul  and  one  in  Chicago) 
utilizing  the  communications  features  of  the  NovaNET  system. 

The  primary  objective  desired  by  both  teachers  was  to  motivate  the  students  to  write.  The 
second  objective  was  to  increase  the  quantity  and  quality  of  writing  by  providing  an  opportunity 
to  communicate  with  Students  at  other  schools.  Writing  to  real  people,  and  using  the  computer 
to  do  so,  promoted  a  greater  amount  of  v/riting  and  a  greater  willingness  to  put  their  thoughts 
in  writing.  The  final  objective  was  to  give  the  students  an  opportunity  to  learn  more  about  a 
very  different  community,  ai^d  more  about  their  own  too! 

The  NovaNET  notesfile  utility  was  very  will  suited  for  our  writing  project.  The  teachers  would 
write  base  notes  to  start  the  discussions  and  students  v/ould  respond  to  the  base  notes  and  to  each 
other. 

The  notes  contained  to  writer's  sign-on  name  aiid  date  and  time  written.  All  students  in  a  group 
could  read  any  of  the  notes  v/ritten  by  members  of  their  group  and  easily  respond  at  any  point 
of  the  discussion. 

The  project  was  will  received  by  the  teachers  and  students  and  the  CERL  staff  has  applied  for 
grants  to  further  the  project  to  include  more  schools.  We  are  also  planning  to  develop 
relationships  with  alumni  of  the  participating  schools  either  in  post-secondary  schools/or  in 
business  or  industry  or  faculty  people  who  could  discuss  their  work  with  the  students  via  the 
network. 
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Using  ERIC  -  A  Database  and  More 

Jane  Janis,  ERIC  Clearinghouse  on  Information  Resources 

When  people  think  of  ERIC  they  think  of  the  ERIC  database  but  the  ERIC  system  provides  a 
lot  more.  Many  users  only  think  or  ERIC  when  they  want  to  get  a  lot  of  information  for  a 
research  project  but  ERIC  is  also  an  important  current  awareness  tool.  In  addition  to  the 
maintenance  of  the  database,  each  clearinghouse  publishes  a  bi-annual  newsletter,  a  series  of 
digests  and  mini-bibliographies,  information  products,  and  a  monograph  series.  There  is  also 
a  users  services  coordinator  in  each  clearinghouse  whose  primary  responsibility  is  helping  users 
meet  their  information  needs. 

The  purpose  of  this  presentation  is  to  talk  about  all  these  aspects  in  more  detail  and  help  those 
attending  the  presentation  see  what  else  ERIC  can  provide  for  them  in  an  ongoing  manner...  not 
just  when  they  need  a  search  done. 
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The  pfsisenter,  Jane  K.  Janis  has  been  with  the  ERIC  Clearinghouse  on  Information  Resources 
for  the  last  ten  years.  In  that  capacity,  she  has  been  part  of  the  continuing  process  of  producing, 
evaluating  and  improving  all  the  different  aspects  of  the  Clearinghouse.  She  has  hands-on 
understanding  of  how  the  system  has  involved  and  all  the  valuable  resources  available  to  its 
users. 
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Effects  of  anonymity  and  group  saliency  on  participation  and 
interaction  in  a  computer-mediated  small-group  discussion 

Terence  C.  Ahem 
Vance  Durrington 
Texas  Tech  University 

Abstract 

Research  on  the  use  of  computer-mediated  in  learning  environments  demonstrates 
that  the  levels  of  member  participation  as  well  as  achievement  are  consistently  high 
when  compared  to  more  traditional  learning  environments.  However,  closer 
scrutiny  this  medium  of  exchange  can  have  a  negative  impact  on  the  ability  of 
groups  to  structure  interaction  across  time.  This  paper  reports  on  the  amount  of 
individual  interaction  in  a  CMC  environment  in  which  one  condition  had  a  high 
level  of  group  identification  in  contrast  to  a  condition  which  maintained  individual 
anonymity. 

Introduction  and  Rationale 

Research  (Kraemer  &  Pinsonneault,  1990)  on  the  use  of  computer-mediated  CMC 
in  learning  environments  demonstrates  that  the  levels  of  member  participation  as 
well  as  achievement  are  consistently  high  when  compared  to  more  traditional 
learning  environments.  Hiltz  (1988)  reports  that  students  who  participate  in 
computer-mediated  classes  participate  at  levels  of  interaction  which  would  not  be 
possible  in  a  traditional  classroom.  Phillips  and  Santoro  (1989)  have  found  that 
CMC  removes  time  and  space  as  a  constraint,  so  that  student-to-student  as  well  as 
student-to-teacher  interaction  is  not  only  convenient  but  also  an  effective 
instructional  strategy. 

However,  on  closer  scnitiny  this  medium  of  exchange  can  affect  the  ability  of 
groups  to  structure  interaction  across  time.  According  to  Turner  ( 1988)  "sustained 
structuring  is  accomplished  through  'chains  of  interaction'  where  in  a  literal  sense, 
individuals  'pick  up  where  they  left  off  from  past  encounters....Without 
structuring,  every  re-encounter  of  individuals  would  involve  so  much  interpersonal 
work  that  they  would  exhaust  themselves."  (p.  121).  This  is  an  important  issue  for 
learning  groups  because  interaction  is  usually  not  confined  to  a  single  encounter. 
Recent  research  has  observed  that  CMC  systems  may  inherently  cause  a  breakdown 
in  the  normative  sequence  of  interaction. 

McGrath  (1990)  observes  that  because  group  members  in  an  asynchronous  system 
can  choose  when  to  participate  an  unpredictable  lag  in  feedback  is  created  which 
dramatically  affects  the  dynamics  of  the  group's  interaction  structure.  According  to 
McGrath  current  computer-mediated  communication  systems  reduce  the  "normative 
force"  (1990,  p.  52);  that  is  the  expectation  that  communicative  events  are  regulated 
and  follow  expected  patterns.  However,  because  of  an  unrestricted  ability  to 
participate  wi^in  a  computer-mediated  system,  any  member  may  interject  a 
comment  completely  out  of  rhythm  with  the  rest  of  the  group.  This  breakdown  in 
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the  anticipated  connectedness  between  messages  (Rogers  &  Kincaid,  1981)  creates 
"anomalies  in  the  transition  of  floor  turns,  and  the  smooth  succession  of 
^speakers'"  (McGrath,  1990,  p.  47)  and  may  restrict  the  amount  of  reciprocal 
interaction  between  members. 

Kiesler  (1986)  theorizes  that  computer-mediated  systems  breakdown  standard 
interaction  because  CMC  attenuates  the  normative  social  context  clues  by  restricting 
the  ability  of  senders  "to  link  the  content  or  tone  of  messages  to  the  receiver's 
responses  so  they  can  evaluate  how  their  messages  are  being  received/  (p.  48). 
Consequently,  without  the  imj)licit  social  information  such  status  or  gender, 
participants  in  CMC  feel  a  greater  sense  of  i>ersonal  anonymity  and  less 
responsibility  to  the  group.  This  results  in  group  members  who  are  unconcerned 
with  meeting  normative  requirements  for  maintaining  interactional  structure  over 
time. 

Lea  (1991 )  on  the  other  hand  speculates  that  the  breakdown  in  reciprocal  interaction 
is  due  to  a  lack  of  group  salience  and  not  the  attenuation  of  social  context  cues. 
According  to  Lea,  participants  identify  strongly  with  the  group  are  more  likely  to 
strive  to  create  norms  for  group  interaction.  In  contrast  a  higher  level  of  anonymity 
reduces  group  salience  and  forces  the  individual  to  invoke  the  social  norms  from  the 
larger  social  context  in  which  the  group  is  embedded.  Consequently,  current  CMC 
systems  reduce  group  salience  by  hampering  the  development  of  consistent 
exchange  patterns.  Because  it  is  more  difficult  in  CMC  environments  to  detennine 
'*who  said  what  to  whom  when"  participants  are  reluctant  to  risk  involvement  which 
results  in  more  conservative  interaction  sequences. 

Problem  Statement 

Many  researchers  (Kiesler,  1986;  Kraemer  &  Pinsonneault,  1990;  McGrath,  1990; 
Murray,  1985)  have  noticed  that  a  breakdown  in  normative  function  reduces 
members  effectiveness  in  using  computer-mediated  interaction  systems.  They 
suggest  that  a  simple  solution  to  the  problem  would  be  to,  as  McGrath  points  out, 
explicidy  create  "the  very  kinds  of  social  norms  that  apparently  arise  spontaneously 
in  natural  face-to-face  groups"  (McGrath.  1990,  p.  55). 

On  the  other  hand,  Hesse,  Werner,  &  Allman  ( 1988)  speculates  that  this 
breakdown  in  nonnative  function  is  a  technological  problem  which  could  be 
resolved  by  designing  new  software  that  could  provide  better  temporal  sequencing 
of  comments.  Recent  research  (Ahem,  1991a)  has  demonstrated  that  the 
breakdown  in  normative  interaction  sequence  can  be  alleviated  through  a  careful 
design  of  the  user  interface. 

Consequently,  we  propose  to  study  the  effects  of  providing  members  who  are 
inteiacting  through  a  CMC  environment  with  external,  face-to-face  group 
identification  in  contrast  to  a  group  which  maintains  individual  anonymity. 
Additionally,  we  will  study  whether  there  is  an  effect  on  the  amount  of  group 
interaction  by  design  of  the  CMC  interface. 
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It  is  expected  that  increasing  group  salience  through  participant  identification  in  a 
traditional  text-based  CMC  environment  will  produce  an  equal  level  of  student 
interaction  in  a  non-traditional  CMC  environment  which  provides  a  graphic 
representation  as  well  as  the  dynamic  linking  of  participant  responses.  It  is 
expected  that  maintaining  participant  anonymity  in  a  traditional  text-based  CMC 
environment  will  produce  an  equal  level  of  student  interaction  in  a  non-traditional 
CMC  environment  which  provides  a  graphic  representation  as  well  as  the  dynamic 
linking  of  participant  responses.  It  is  expected  that  increasing  group  salience  in  a 
graphic-based  CMC  environment  through  participant  identification  will  produce  an 
equal  level  of  student  interaction  in  a  graphic-based  CMC  environment  where  the 
anonymity  of  participants  is  maintained.  Further  it  is  expected  that  increasing 
group  salience  in  a  traditional  text-based  CMC  environment  through  participant 
identification  will  produce  an  equal  level  of  student  interaction  in  a  traditional  text- 
based  CMC  environment  where  the  anonymity  of  participants  is  maintained. 

Subjects 

The  subjects  in  the  study  were  15  undergraduate  students  from  an  introductory 
course  in  Office  systems  technology  in  a  southwestern  public  university. 

Materials 

The  IdeaWeb®  (Ahem,  1991b)  is  software  designed  and  developed  by  the  author 
using  HyperCard  to  mediate  group  discussion.  The  software  has  three  levels, 
Topic,  Discourse  and  Comment .  There  are  two  versions  of  the  software  which 
differ  at  the  discourse  level.  For  each  version,  in  order  to  respond  to  a  subtopic  a 
user  will  click  the  button  labeled  Discussion  . 

Version  one  employs  a  graphic-based  interface  at  the  discourse  level  which  requires 
group  members  to  visually  map  their  interactions.  In  this  version  when  a  student 
adds  a  statement  to  the  discussion,  the  computer  system  automatically  creates  a 
Comment  Icon  and  labels  it  with  the  respondents  name  and  the  day's  date.  Once 
the  student  has  made  their  comment  and  returned  to  the  graphic  map,  the  new 
Comment  Icon  must  be  moved  into  the  map  area  where  it  can  be  linked  to  previous 
responses.  Linking  comments  creates  not  only  a  visual  representation  but  also  a 
dynamic  response  cluster.  Therefore  in  browsing  previous  responses,  a  selected  a 
node  within  an  interaction  sequence  would  be  displayed  first  and  subsequently  any 
other  response  in  the  order  in  which  it  was  linked  with  to  the  primary  node. 

Version  two  of  the  IdeaWeb  employs  a  more  traditional  textual  interface  in  which 
the  participants  see  a  sequential  list  of  member  comments.  To  add  a  comment  in 
version  two  the  system  requests  the  name  of  the  addressee  and  automatically 
concatenates  it  with  the  name  of  the  current  user.  The  computer  appends  this 
information  to  the  end  of  a  chronological  list  of  previous  respondents.  To  read 
comments  in  this  version,  the  student  would  select  the  comment  by  clicking  on  the 
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appropriate  line.  The  computer  displays  the  selected  comment.  The  computer 
system  provides  users  with  an  opportunity  to  review  other  comments  in  in 
chronological  order  starting  from  the  selected  response.  Within  version  two, 
discussion  respondents  do  not  have  any  method  of  altering  the  order  in  which 
responses  are  displayed. 

Treatments 

The  subjects  were  randomly  divided  into  4  four  conditions:  group  salience  with 
graphic-based  interface,  group  salience  with  text-based  interface,  pen-name  with 
graphic-based  interface  and  pen-name  with  text-based  interface  of  approximately  4 
members  per  group.  The  group  salience  conditions  met  face-to-face  for 
approximately  30  minutes  prior  to  the  start  of  the  computer-mediated  discussion  and 
will  orally  exchange  information,  such  as  occupation  and  age.  The  pen-name 
conditions  did  not  meet  outside  of  the  computer-mediated  system  which  generated 
aliases  for  each  member  of  the  discussion  group. 

Over  a  period  of  approximately  3  weeks  all  conditions  participated  in  an  open 
discussion  based  on  the  topics  of  security,  privacy,  education  and  regulation  in  the 
telecommunications  industry..  Due  to  the  nature  of  this  study,  data  such  as  the 
number  of  messages  generated  and  time  will  be  automatically  collected  from  the 
actions  of  the  participants  in  the  compufer-mediated  groups. 

Analysis 

A  significant  main  effect  for  identity  was  found  (Wilks  1  =  .299,  F  (4,8)  =  4.686, 
p  =  .030).  Post  hoc  univariate  F  tests  revealed  that  that  members  in  the 
anonymous  condition  significantly  (F(l,l  1)  =  19.439,/?  =  .001)  wrote  more 
words  (i4  =  64.8)  per  message  than  the  salient  condition  (/<  =  1 1 .8).  Additionally, 
anonymous  groups  significantly  (F(l,l  1)  =  1601.1, p.  ,008)  spent  longer  = 
22.4  minutes)  per  message  in  contrast  to  the  salient  group  (//  =  1 .6  minutes). 

Discussion 

The  typical  classroom  interaction  as  characterized  by  Mehan  (Mehan,  1979)  is  a 
discourse  sequence  of  initation-reply-evaluation.  This  type  of  discourse,  more 
traditionally  referred  to  as  recitation  (Cazden,  1988;  Dillon,  1984)  has  die  teacher 
controlling  who  talks  about  what  to  whom,  which  results  in  the  teacher  talking  three 
times  more  often  than  students.  The  typical  way  for  a  student  to  have  an 
opportunity  to  interact  is  usually  in  response  to  a  direct  question  posed  by  a  teacher 
who  then  evaluates  its  correctness.  Finally,  the  recitation  model  of  classroom 
interaction  also  limits  the  opportunity  for  students  to  interact  with  their  peers. 

Dillon  (1988)  reports,  however,  that  the  traditional  classroom  interaction  is  contrary 
to  the  more  natural  situation.  He  observed  students  on  the  playground  where  the 
number  of  questions  and  general  interaction  with  peers  was  govemed  by  the  age  of 
the  interactants.  The  older  they  were,  the  more  complex  and  extensive  the 
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interaction,  coupled  with  a  more  sophisticated  world  view.  Dillon  was  surprised  to 
find  that  in  the  classroom,  this  situation  was  reversed.  Instead  of  the  older  students 
interacting  more  often  with  their  peers  they  made  fewer  and  fewer  comments  and 
asked  fewer  and  fewer  questions  as  they  moved  through  school.  Dillon  suggests 
that  the  interaction  sequence  of  initiation-reply-evaluation  trains  students  to  respond 
with  only  enough  information  necessary  to  appropriately  answer  the  question. 
Students  learn  very  early  that  it  does  not  benefit  them  at  all  to  volunteer  or  elaborate 
an  answer  to  a  question  unless  they  know  exactly  what  the  teacher  expects.  They 
realize  that  the  content  of  the  response  is  valued  and  not  the  exploration  of  ideas. 
The  findings  in  this  study  support  this  ob: .  p/ation.  The  students  in  the  salient 
condition  were  not  willing  to  risk  longer  or  .nore  complex  interaction.  However, 
under  the  protection  of  anonymity  the  comments  became  longer  and  the  students 
spent  longer  time  engaged  with  the  material.  This  has  important  consequences  for 
instmction.  It  is  simply  not  enough  to  provide  students  with  the  opportunity  for 
peer-interaction  by  employing  computer  technology,  but  the  traditional  role  of  the 
teacher  as  judge  must  also  be  altered  in  meaningful  ways  if  students  will  risk  further 
interaction. 
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Enhancing  Context  and  Continuity  in  Computer- mediated 
Conferencing:  Developing  the  IdeaWeb^ 


Terence  C.  Ahern 
Texas  Tech  University 

Abstract 


Research  on  the  use  of  computer-mediated  communication  (CMC)  in  instruction  has 
demonstrated  that  user  participation  as  well  as  achievement  are  consistently  high 
when  compared  to  more  traditional  learning  environments.  Previous  investigation, 
however,  has  revealed  that  computer-mediated  communication  systems  can  reduce 
the  "normative  force"  that  exists  in  face-to-face  discourse  for  stmcturing  a  group^s 
interaction.  This  results  in  higher  user  disorientation  and  a  breakdown  in  the 
sequence  of  interaction  in  CMC.  Attempts,  at  reducing  this  effect  have 
concentrated  on  developing  text-based  systems,  which  seek  to  constrain  user 
responses.  However,  this  article  reports  on  the  development  and  implementation  of 
a  graphic-based  user  interface  that  reduced  the  breakdown  in  the  interaction 
sequence  in  a  computer-mediated  small-group  discussion. 


Introduction 

For  a  long  time,  computer  technology  has  been  viewed  as  playing  a  "critical 
role  in  the  transformation  of  the  existing  educational  systems"  (Hannafin  &  Peck, 
1989,  p.  v).  Even  though  the  computer  has  not  resulted  in  a  radical  revision  of  the 
school,  Hannafin  and  Peck  report  that  "progress  is  being  made"  (p.  vi).  One  area 
in  which  innovation  is  evident  is  telecommunication. 

Watson  (1990)  reports  that  "through  the  commitment  of  individual  teachers, 
the  American  classroom  is  going  on-line"  (p.  1 10).  Both  teachers  and  students  are 
tapping  into  a  vast  array  of  people  and  information.  Electronic  bulletin  boards,  the 
modem  equivalent  of  a  town  meeting,  are  springing  up  on  almost  any  topic  or 
subject  Schools  in  one  state  can  interact  with  schools  in  other  states  and  in  some 
cases  other  countries.  In  addition,  the  use  of  electronic  telecommunication  is  not 
limited  to  chit-chat  among  pen-pals,  but  whole  courses  are  being  delivered  on-line 
(Harasim,  1990). 

Research  on  the  use  of  computer-mediated  communication  (CMC)  in 
learning  environments  (Ahem,  Peck,  &  Laycock,  1992;  Hiltz,  1988;  Kraemer  & 
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Pinsonneault,  1990;  Phillips  &  Santoro,  1989;  Quinn,  Mehan,  Levin,  &  Black, 
1983)  has  demonstrated  that  the  levels  of  member  participation  as  well  as 
achievement  are  consistently  high  when  compared  to  more  traditional  learning 
environments.  Phillips  and  Santoro  (1989)  found  that  CMC  removes  time  and 
space  as  a  constraint,  so  that  student-to-student  as  well  as  student-to-teacher 
interaction  is  not  only  convenient  but  an  effective  instructional  strategy.  Hiitz 
(1986)  also  reported  that  students  who  participate  in  computer-mediated  classes 
participate  at  levels  of  interaction  that  would  not  be  possible  in  a  traditional 
classroom. 

Nonetheless,  the  bulk  of  this  research  has  been  limited  to  task-oriented 
groups  normally  found  in  an  office  environment  (Greif,  1988).  Greif  cautions  us 
thi  I  "specific  classes  seen  in  the  office  may  not  exhaust  all  the  kinds  of  differences 
an(  mteractions  that  can  be  found  among  classes  of  people  working  together  in 
other  settings"  (1988,  p.  11).  Consequentiy,  even  though  CMC  has  been  shown  to 
have  important  effects  on  the  level  of  user  participation  within  groups,  few  studies 
have  looked  at  how  a  computer-mediated  environment  affects  gioup  dynamics 
(Ahem,  et  al.,  1992;  Kraemer  &  Pinsonneault,  1990). 

Small  groups  create  an  environment  that  encourages  a  fluid  interchange 
among  its  inembers  where  "each  person  influences  and  is  influenced  by  each  other 
person"  (Shaw,  1976,  p.  1 1).  Even  though  most  groups  exhibit  a  high  amount  of 
interchange,  there  is  an  orderly  pattern  or  structure  which  regulates  the  interactions 
among  members  and  provides  rules  for  "what  behaviors  a  group  considers 
appropriate  or  inappropriate"  (Shimanoff,  1988,  p.  50). 

Structure  provides  the  necessary  "stability,  regularity,  and  predictability  to 
the  system"  (Rogers  &  Kincaid,  1981,  p  146)  without  which  "every  reencounter  of 
individuals  would  involve  so  much  interpersonal  work  that  they  would  exhaust 
themselves"  (Turner,  1988,  p.  121).  Consequendy,  group  members  are  able  to 
form  context  by  creating  "'chains  of  interaction'  where  in  a  literal  sense,  individuals 
'pick  up  where  they  left  off  from  past  encounters"  (Turner,  1988,  p.  121). 
Therefore,  "as  the  discourse  proceeds,  it  carries  and  builds  with  it  a  basis  of  shared 
understandings  in  terms  of  which  the  participants  are  able  to  make  joint  sense  of 
what  they  are  saying  and  doing"  (Edwards  &  Mercer,  1989,  p,  93). 

Typically  CMC  is  an  asynchronous  form  of  discourse  which  introduces  a 
temporal  lag  between  the  transmission  and  the  reception  of  a  message.  This 
temporal  lag  can  have  a  dramatic  impact  on  the  ability  of  groups  to  structure 
interaction  across  time  (Ahem,  1991;  McGrath,  1990).  Current  CMC  designs  are 
not  suited  for  the  multi-channel  interaction  common  of  small  group  interaction. 

The  standard  user  interface  design  duplicates  tlie  office  memo  (Comer  & 
Pederson,  1985)  which  identifies  to  whom  a  message  was  addressed,  from  whom 
it  originated,  the  date  it  was  sent,  as  well  as  simple  topic  information.  Outside  of 
displaying  a  chronological  list  of  messages  to  a  reader,  there  are  no  visual  or 
semantic  cues  which  might  illustrate  a  fundamental  reladonship  among  the 
messages.  This  lack  of  orienting  cues  can  make  it  difficult  for  a  user  to  arrange 
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messages  in  the  proper  order  necessary  to  reconstruct  an  interaction  sequence, 
especially  when  the  "messages  arise  from  more  than  one  source"  (McGrath,  1990, 
p.  52).  Consequently,  a  group  member  can  become  easily  disoriented  because  a 
relevant  comment  was  missed  or  read  out  of  the  proper  sequence.  In  other  words, 
according  to  McGrath,  current  computer-mediated  communication  systems  reduce 
the  "normative  force"  which  is  the  expectation  of  an  orderly  pattern  for  the  group's 
interaction  (McGrath,  1990,  p.  52).  Therefore,  a  reduction  in  the  "normative 
force"  among  group  members  causes  a  breakdown  in  the  expected  relationship 
benveen  messages  (McGrath,  1990;  Rogers  &  Kincaid,  1981 )  which  may  restrict 
the  amount  of  reciprocal  interaction  between  members.  Consequently,  the  ability  of 
CMC  to  provide  groups  with  an  unrestricted  flexibility  to  communicate  may  in  fact, 
be  its  greatest  weakness  for  efficient  group  interaction. 

One  solution  to  the  problem  of  structural  breakdown  would  be,  as  McGrath 
points  ouU  to  create  externally  "the  very  kinds  of  social  norms  that  apparently  arise 
spontaneously  in  natural  face-to-face  groups"  (Ahem,  et  al.,  1992;  McGrath,  1990, 
p.  55).  On  the  other  hand,  Hesse, Werner  and  Altman  (1988)  suggest  that  this 
breakdown  in  standard  interaction  is  a  technical  problem  which  could  be  solved  by 
developing  new  software  that  could  provide  better  temporal  su^uencing  of 
comments. 

Previous  attempts  by  researchers  to  solve  the  technical  problems  in  the 
seemingly  disorderly  world  of  asyiichronous  computer-mediated  communication 
have  concentrated  on  developing  software  routines  that  structure  a  group's 
interaction.  Comer  and  Pederson  (1985)  designed  and  implemented  a  software 
package  for  asynchronous  CMC  based  on  the  assumption  that  the  content  of  a 
specific  message  is  better  understood  within  a  context  of  messages  that  are  grouped 
by  topic.  Winograd  (1988)  also  has  developed  a  system  based  on  the  notion  that 
interaction  is  a  "coordinated  sequence  of  acts"  (p.  628).  Winograd  assumed,  as 
did  Comer  and  Pederson,  that  a  message  could  be  understood  only  in  relationship 
to  other  messages.  Consequently,  the  system  designed  by  Winograd,  provided 
users  with  a  variety  of  underlying  constraints  designed  to  structure  interaction. 

These  earlier  attempts  to  improve  the  user's  response  behavior  in 
computer-mediated  systems  have  concentrated  on  a  text-based  environment 
Cognitive  psychology,  however,  has  learned  that  providing  a  visible  framework  or 
structure  improves  the  integration  of  the  conceptual  relationships  among  the  ideas 
presented.  One  such  approach  has  been  to  map  the  relationships  among  ideas 
visually  through  the  use  of  a  class  of  cognitive  sti*ategies  called  graphic  organizers. 
Consequently,  with  the  current  graphic  abilities  of  modem  microcomputing 
systems,  incorporating  a  more  visually  informative  interface  may  reduce 
disorientation  in  a  computer-mediated  discussion  by  providing  the  group  members 
with  a  contextualizing  structure  of  the  current  interaction. 

Therefore,  the  primary  purpose  of  this  paper  is  to  report  on  tlie  effect  a 
redesign  of  the  user  interface  design  had  on  the  amount  of  reciprocal  interaction 
between  group  members.  It  is  anticipated  that  the  breakdown  in  normative 
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interaction  structure  encouraged  by  the  asynchronous  nature  of  computer-mediated 
systems  can  be  obviated  through  a  graphic  design  of  the  user  interface. 

Designing  the  IdeaWeb^ 

The  IdeaWeb  software  was  designed  and  developed  using  Apple 
Computer's  HyperCard™  to  mediate  small  group  discussion.  The  software  has 
three  levels:  a  topic  map,  a  discussion  map,  and  individual  comment  cards.  The 
topic  map  (see  Figure  1)  is  tlie  first  level  that  a  user  encounters  after  logging  onto 
the  software.  This  design  approach  was  suggested  by  Comer  and  Pederson 
(1985),  who  argued  that  grouping  messages  according  to  topic  would  orient  users 
thereby  improving  member  response  patterns. 

At  the  topic  map  level,  a  user  selects  a  subtopic  by  highlighting  a  Topic  Icon 
which  in  turn  displays  a  short  descriptive  blurb  of  the  subtopic  in  the  area  to  the 
right  of  the  map. 


Forced  or  Volimt&zy  School 
latejCFation 


The  Meighborhoods 

Without  adequate  legislation  to 
integrate  communities, 
integration  ends  when 
students  return  to  their 
respectLw  neighborhoods  at 
the  close  of  the  school  day. 
Little  is  being  done  to  insure 
that  underclass  and  minority 
families  are  protected  from 
discriminatory  housing 
practices, 


Figure  1:  Topic  Map  with  subtopic  selected 
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In  the  current  implementation,  subtopic  descriptions  were  entered  prior  to  the 
discussion.  Once  a  student  has  selected  a  subtopic  to  explore,  moving  to  the 
appropriate  Discussion  Map  is  accomplished  by  a  simple  clicking  of  the 
"Discussion"  button  located  in  the  lower  right  part  of  the  screen. 

Each  subtopic  has  a  Discussion  Map  which  moderates  the  creating, 
browsing  or  linking  of  individual  comments.  In  addition  the  Discussion  Map 
requires  each  user  to  chart  their  interactions  visually  (see  Figure  2). 
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Figure  2:  Graphic-based  discussion  screen  with  new  Comment  Icon 


To  add  a  comment  to  a  ^  articular  discussion,  a  user  would  click  the  Comment 
button  located  in  the  lower  left  comer  of  the  Discussion  Map  screen.  The  computer 
software  automatically  creates  a  new  Comment  Icon  in  the  "New  Comment"  box 
and  labels  it  with  the  user's  name  and  the  current  date.  The  IdeaWeb  then 
immediately  displays  a  blank  comment  card  (see  Figure  3)  ready  to  accept  the  user's 
note. 
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Figure  3:  Typical  comment  card  with  response 

Once  a  user  has  finished  the  current  comment  and  returned  to  the  Discussion  Map, 
the  new  Comment  Icon  must  be  repositioned  in  the  map  area.  The  user  places  the 
cursor  on  the  new  Comment  Icon  and  while  holding  down  the  mouse  button  moves 
the  cursor  somewhere  on  the  discussion  map.  Once  the  user  releases  the  mouse 
button,  the  software  positions  the  Comment  Icon  at  the  last  location  of  die  cursor. 

Comment  Icons  also  form  the  nodes  in  an  iuf'-raction  sequence  through  the 
process  of  linking.  To  link  a  comment  to  another  comment,  the  user  selects  bodi  a 
scarce  icon  and  a  target  icon  as  illustrated  in  figure  4. 
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Figure  4:  Selecting  a  source  icon  and  target  icon  for  linking 

Once  each  node  has  been  selected  the  user  clicks  the  Link  button  which  causes  the 
system  to  draw  a  line  that  visually  links  the  two  nodes  together.  Simultaneously  the 
software  identifies  the  identification  number  of  each  icon  dx\d  stores  this  information 
as  a  double-linked  anay  which  the  system  uses  during  browsing. 

In  order  to  read  previous  responses,  a  group  member  would  select  a  source 
node  by  highlighting  a  particular  Comment  Icon.  By  clicking  the  Browse  button 
located  at  the  bottom  of  the  Discussion  Map,  the  IdeaWeb  displays  the  selected 
s.ource  node  first  and  then,  with  a  simple  click  of  the  mouse  button,  any  additional 
nodes  in  tlie  order  in  which  they  were  linked.  The  user  is  returned  to  the 
Discussion  Map  once  the  sequence  of  links  has  been  exhausted.  However,  the 
software  allows  the  user  to  interrupt  a  display  sequence  by  issuing  a  key/mouse- 
click  command  string  which  immediately  returns  the  user  to  the  Discussion  Map  at 
any  time  during  browsing. 

Once  the  user  has  finished  with  a  particular  subtopic,  the  user  would  return 
to  the  Topic  Map  by  clicking  the  Topic  button.  The  user  could  now  select  a 
different  subtopic  or  exit  the  program  by  clicking  the  Quit  button. 
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Implementation 

More  often  than  not,  the  instructional  tactics  of  the  modem  classroom 
follow  the  traditional  patterns  of  lecture/recitation  (Cazden,  1988;  Mehan,  1979) 
which  Dillon  characterizes  as  "the  familiar  form,  ...(among  other  aspects)  of 
recurring  sequences  of  teacher  questions,  plus  student  answers,  where  students 
*recite'  what  they  already  know  or  are  coming  to  know  through  the  questioning" 
(Dillon,  1984,  p.  50).  Recitation  is  teacher  directed  and  teacher  controlled  with  few 
opportunities  for  students  to  initiate  an  interaction  and  even  fewer  chances  to 
interact  with  peers. 

Many  researchers  (Cazden,  1988;  Johnson  &  Johnson,  1990;  Sharan  & 
Sharan,  1976)  agree  that  there  is  a  need  to  shift  classroom  interaction  away  from  the 
current  paradigm  of  recitation,  where  only  the  teacher  controls  the  interaction.  In 
contrast  to  the  lecture/recitation,  the  classic  small-group  discussion  seeks  a  '^fuller 
understanding,  a  wider  grasp  of  information  pertinent  to  a  topic,  or  consideration  of 
a  problem  from  as  many  points  of  view  as  possible"  (Brilhart,  1986,  p.  323).  The 
classroom  discussion  group  is  a  collection  of  individuals  who  are  jointly  seeking  a 
deeper  understanding  of  a  work  of  art,  an  historical  event,  or  a  political  policy  from 
as  many  different  perspectives  as  possible. 

Nonetheless  the  standard  interaction  pattern  has  persisted  in  the  modem 
school  because  it  can  be  very  efficient  in  dispensing  large  amounts  of  information  in 
the  least  amount  of  time.  Small-group  discussion  on  the  other  hand,  takes  a  great 
deal  of  time  to  execute  -  much  more  than  traditional  recitation  -  because  it  involves 
a  many-to-many  interaction  sequence  instead  of  the  more  direct  one-to-many  typical 
of  today's  classroom.  Computer-mediated  communication,  however,  as  Phillips 
and  Santoro  (Phillips  &  Santoro,  1989)  have  demonstrated,  can  provide  a 
convenient  and  effective  system  for  both  student-teacher  and  studeni-student 
interaction  by  removing  the  constraints  of  time  and  space.  This  has  important 
implications  for  designing  instruction. 

The  Idea  Web  was  tested  (for  a  complete  report  see  Ahem,  1991)  against  the 
more  familiar  text-based  conferencing  system  within  a  small-group  discussion. 
Sixty-five  students  in  a  large  undergraduate  introductory  educational  theory  and 
policy  course  at  a  large  public  university  volunteered  to  participate  in  the  study. 
Each  condition  was  random.ly  assigned  to  a  particular  interface,  so  that  each  small 
group  would  be  contain  approximately  5  members.  This  resulted  in  the  text-based 
condition  having  6  discussion  groups  while  the  graphic  based  condition  having  7 
groups.  The  course  instructor  selected  a  topic  of  general  discussion  which  was 
related  to  a  particular  unit  the  class  as  a  whole  was  currently  investigating. 

Students  in  each  of  the  groups  had  only  two  possible  audiences  to  whom  a 
message  could  be  addressed:  either  to  the  group  as  a  whole  or  to  a  specific 
individual.  There  was  a  statistically  significant  effect  (X^  (1,  N=441)  =  5.30,  £ 
=.021)  on  the  audience  selection  by  type  of  interface.  The  audience  of  choice  in  the 
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graphic-based  interface  was  a  more  often  peer  in  contrast  to  the  text-based 
condition.  One  student  offered  the  following  assessment  of  the  graphic  interface. 


[The  graphic-based  interface]  made  it  [the  interaction]  into  a 
conversation,  like  we  were  reading  a  dialogue  between  people. 


This  observation  is  supported  by  the  following  student  comment,  which  seems  to 
indicate  that  the  linking  of  messages  within  the  IdeaWeb  helped  users  to 
contextualize  previous  messages  into  an  active  interaction  sequence  prior  to 
responding  with  a  comment  of  their  own  . 


S:  I  liked  it  because  you  got  to  find  out  what  other  people  thought 
and  it  stimulated  something  that  I  was  thinking  [about]  and  I  could 
relate  what  I  was  thinking  to  what  everyone  eise  was  thinking  before 
I  put  [in]  m y  conunent  I  would  read  one  comment  and  it  would 
[be]  a  long-winded  paiagraph.  Then  somebody  else  would  say  I 
really  agree  with  that  person.  Then  I  could  either  disagree  or  agree. 
Finally  it  was  like  an  on  going  debate. 


Martson  and  Hecht  (Martson  &  Hecht,  1988)  suggest  that  group  members 
will  participate  more  often  if  they  perceive  their  comments  have  value,  which  they 
define  as  attending  to  what  someone  else  is  saying  in  an  interaction.  In  a  computer- 
mediated  system,  the  notion  of  value  would  be  demonstrated  by  directing  a 
response  to  another  member's  previous  comment.  For  example,  the  following 
message  was  left  by  a  student  using  the  graphic -based  interface. 


Eric 

11/30/90  1:20  PM 

Total  words  =  22 

Yes,  pluralism  is  needed!!!! 

By  die  way,  my  last  comment  is  lonely  over  there,  it  hasn't  been 
linked  to  anyone!! 


Students  in  the  graphic  interface  on  average  produced  more  messages 
(|l=5.088)  than  students  in  the  textual  interface  (^1=  3.677),  as  seen  in  Table  2 
which  was  statistically  significant  (E,  (1,63)  =  4.564,  ^  =  .02935).  The 
graphical  charting  of  remarks  in  the  IdeaWeb  provided  users  with  immediate  visual 
feedback  concerning  any  particular  response  to  an  individual  comment  and  seems  to 
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have  encouraged  members  to  interact  more  frequently  in  contrast  to  the  more 
traditional  text-based  interface. 

Future  Design  Improvements 

The  Idea  Web  software  as  tested  initially  by  having  the  participants  check  out 
a  single  diskette,  which  could  be  used  at  any  campus  location.  According  to  the 
study,  this  style  of  implementation  did  not  cause  any  access  problems  for  the 
students.  However,  the  next  step  in  development  would  be  to  test  its  effect  in  a 
distributed  local  area  network,  such  as  AppleTalk. 

Further,  an  instructor/gatekeeper  controls  the  creation  of  subtopics  in  the 
current  implementation  of  the  IdeaWeb.  One  refinement  would  be  to  allow  for  the 
dynamic  creation  of  topics  and  subtopics  by  users  in  an  interactive  fashion. 


Summary 

The  question  that  this  software  design  investigated  was  whether  the  usual 
interaction  structure  in  CMC  systems  could  be  improved  by  employing  a  graphic- 
based  design  of  the  user  interface.  The  development  and  implemenmtion  of  the 
IdeaWeb  demonstrates  that  it  is  possible  to  design  a  system  that  encourages 
exchanges  customary  to  small-group  discussions  without  hampering  the  group's 
interaction.  Such  a  system  has  important  implications  for  instruction. 

Tlirough  a  simple,  standalone  microcomputer  system,  instructors  can 
provide  opportunities  for  learners  to  interact  not  only  with  the  teacher  but  with  their 
peers  much  more  frequently.  Students  will  have  the  opportunity  to  challenge  their 
understanding  of  course  material  to  a  greater  depth  or  to  explore  issues  tangential  to 
the  main  instructional  objective.  Further  a  CMC  system  provides  participants  with 
the  immediacy  of  a  real-time  interaction  unrestricted  by  time  and  space,  which  is  not 
possible  using  the  more  traditional  classroom  media  of  paper  and  pencil.  Teachers 
and  students  have  access  to  more  people  over  a  larger  distance  in  a  shorter  amount 
of  time.  Finally,  CMC  technology  provides  a  powerful  but  flexible  tool  for 
organizing  and  retrieving  information  that  provides  students  with  the  ability  to 
explore  not  only  who  said  what  and  when,  but  also  the  context  in  which  the 
interaction  was  embedded. 

Computer  mediated  communication  has  the  potential  to  revolutionize 
instruction  for  many  students.  However,  achieving  more  effective  student 
engagement  in  CMC  is  essentially  a  problem  of  design,  either  redesign  the  user 
interface  or  provide  clear  external  guidelines  for  student  discourse  in  order  to 
encourage  more  group  interaction. 

Therefore  it  is  essential  that  researchers  not  only  investigate  the  effects  of 
CMC  in  a  variety  of  instructional  settings  but  also  the  impact  that  CMC  systems 
have  on  the  dynamics  of  group  interaction.  For  example,  what  is  the  effect  of  a 
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graphic-based  interface  on  the  patterns  of  interaction  in  which  the  course  instructor 
takes  more  active  leadership  role  such  as  gatekeeper/moderator  see  (Ahem,  et  al., 
1992).  Further  what  is  the  effect  of  the  interface  design  if  the  group's  goal  is  more 
directed,  such  as  in  a  problem-solving  group.  Therefore,  investigating  the  effects 
of  CMC  on  group  dynamics  can  provide  important  information  to  instructional 
designers,  software  engineers  and  educators  in  order  to  develop  appropriate 
strategies  for  the  implementation  of  CMC  systems  in  a  myriad  of  learning 
environments. 
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NETWORKS  AND  MULTIMEDIA  SYSTEMS 


Greg  Baur,  Western  Kentucky  University 

ABSTRACT 

The  first  part  of  the  paper  deals  with  the  nature  of  multimedia  systems  and  how  networks  can 
be  used  to  link  individual  multimedia  workstations.  The  major  problem  is  full  motion  video 
which  is  described  in  the  second  part  of  the  paper.  Alternative  solutions  are  discussed  with 
emphasis  on  compression/decompression  algorithms.  Finally,  the  digital  video  interactive  (DVI) 
and  Quick  Time  software  solutions  are  examined. 

IiVTRODUCTION 

During  the  past  several  years,  educators  and  trainers  have  found  that  the  use  of  multimedia  has 
a  tremendous  impact  on  the  learner  when  used  in  an  interactive  environment.  The  notion  of 
multimedia  is  not  a  new  one;  motion  pictures  have  been  used  by  teachers  for  years»  Requiring 
learners  to  use  multiple  senses  has  always  been  appropriate. 

Historically,  however,  the  learning  environment  has  been  a  passive  one;  that  is,  the  learner  would 
watch  and  listen  as  instructional  materials  were  presented.  The  exception  to  this  environment 
was,  of  course,  the  laboratory  where  the  learner  could  experiment  and  be  more  actively  involved 
in  the  learning  experience.  The  use  of  the  laboratory  environment  was  basically  confined  to 
studies  in  the  sciences  and  so  many  learners  had  little  or  no  exposure  to  it.  The  lecture  method 
of  presenting  instruction  was  the  major  technique  used  and  for  some  learners,  this  was 
acceptable.  But  for  many  learners,  the  lecture  was  less  than  satisfactory. 

Successful  teachers  and  trainers  were  those  who  actively  involved  their  learners  in  the  learning 
experience.  Learner  motivation  was  higher  and  the  learners  usually  achieved  at  a  higher  level 
than  under  the  lecture  method. 

During  the  1980's,  the  development  of  the  microcomputer  made  computer-assisted  instruction 
(CAI),  among  other  names,  a  ft.  sible  method  for  delivering  instruction  to  learners.  There  was 
some  interaction  between  the  learner  and  the  computer  but  the  general  quality  of  the 
instructional  materials  was  limited  because  of  the  technical  limitations  of  the  computer. 

Instructional  materials  delivered  by  the  computer  were  textual  and/or  animated  in  nature.  Audio 
and  video  were  not  viable  options. 

The  primary  reason  that  audio  and  video  were  not  available  was  because  they  were  both  analog 
media  and  could  not  be  displayed  on  the  same  screen  at  the  same  time  with  computer-generated 
material.  This  problem  was  partially  solved  with  the  development  of  the  Level  III  videodisk 
system  and  the  accompanying  graphics  overlay  adapter  card  which  permitted  video  from  the 
videodisk  and  graphics  from  the  computer  to  be  displayed  on  the  screen  simultaneously.  This 
was  accomplished  usually  by  a  hardware/software  combination  that  converted  the  computer 
graphics  into  analog  data.  Audio,  however,  was  still  not  feasible  in  digital  format  in  its  natural 
form  such  as  the  human  voice. 
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A  second  limitation  to  having  audio  and  video  on  the  computer  was  the  lack  of  available 
immediate  memory.  A  single  picture  or  a  few  seconds  of  human  voice  required  a  great  deal  of 
memory  in  the  digital  form.  This  limitation  has  been  partially  overcome  by  the  development  of 
relatively  inexpensive  immediate  memor)'  (RAM)  in  the  area  of  four  to  eight  megabytes  and  the 
hard  disk  in  the  area  of  twenty  to  several  hundred  megabytes.  Still  frame  video  and  audio  were 
now  feasible  and  could  be  used  in  CAl  applications. 

There  remained  (and  still  remains)  the  problem  of  full  motion  video.  As  was  pointed  out 
previously,  digitized  video  requires  a  great  deal  of  memory,  for  each  frame  (screen).  Motion 
video,  to  be  acceptable,  requires  that  frames  be  played  back  at  a  rate  of  thirty  frames  per  second 
and  is  marginally  acceptable  at  twenty  frames  per  second.  Today's  technology  simply  does  not 
permit  that  kind  of  transfer  rate  from  hard  disk  (RAM  is  too  small)  to  a  full  screen.  Somehow, 
the  amount  of  storage  needed  for  video  must  be  reduced. 

Since  several  learners  may  wish  to  use  the  same  instructional  materials  simultaneously,  it  is 
natural  to  think  of  using  a  local  area  network  (LAN)  to  link  several  workstations  to  a  file  server. 
There  is  an  additional  reason  to  think  about  the  use  of  networks.  If  instructional  material  is 
available  on  a  videodisk  or  a  CD-ROM,  then  each  workstation  must  have  its  own  videodisk 
player  and  CD-ROM  drive,  thus  increasing  the  cost  of  equipment.  By  networking  the 
workstations,  xUc  contents  of  the  videodisk  and/or  the  CD-ROM  become  a  shared  resource. 
However,  the  shared  resource  solution  creates  new  problems.  Solving  these  problems  is  the 
focus  of  this  paper. 

The  first  part  of  the  paper  will  describe  how  networks  can  be  used  in  the  development  of  a 
multiple  workstation  instructional  delivery  systems.  Advantages  and  limitations  of  networks  in 
this  environment  will  be  presented.  It  will  also  describe  how  voice  and  video  data  are  passed 
by  a  network. 

In  the  second  pard  of  the  paper,  the  problem  of  transferring  full  motion  video  across  a  network 
will  be  discussed.  Alterative  solutions  to  the  problem  will  be  presented. 

The  third  part  of  the  paper  will  give  an  overview  of  two  promising  solutions  to  the  full  motion 
video  problem.  The  digital  video  interactive  (DVI)  technology  and  its  use  of  video  compression 
will  be  discussed.  The  Quick  Time  software  will  also  be  discussed. 

NETWORKING  AND  MULTIMEDIA 

Multimedia  systems  in  education  and  training  are  designed  to  be  used  by  either  a  single  learner 
or  multiple  users  simultaneously.  The  single  user  model  is  good  when  there  are  either  a  small 
number  of  users  or  the  cost  of  a  single  workstation  plus  any  peripherals  makes  having  multiple 
workstations  impractical.  On  the  other  hand,  if  there  are  numerous  users  then  the  availability 
of  multiple  workstations  is  desirable  for  maximum  benefit  to  the  greatest  number  of  users  at  one 
time.  Ideally,  there  would  be  one  workstations  for  each  learner  at  all  times  but  this  situation  is 
seldom  possible  when  costs  are  a  prime  consideration,  as  is  usually  the  case. 
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So  where  possible,  multiple  workstations  are  desirable.  In  order  to  be  able  to  share  the 
instructional  materials  among  several  learners,  the  use  of  a  network  and  a  control  file  server  is 
desirable.  Learners  would  download  portions  of  the  instructional  materials  to  their  workstations 
and  work  on  the  assigned  tasks.  A  group  of  learners  could  work  at  ihe  same  workstation  or  at 
different  workstations  in  a  small,  proxsimitous  area.  Since  the  instructional  materials  would  be 
self-contained  on  the  file  ser\'er,  no  additional  peripherals  would  be  needed  at  each  workstation. 

A  primary  advantage  of  this  type  of  networked  system  is  its  cost.  Good,  reliable  LANS  are 
relatively  inexpensive,  especially  when  the  cost  of  duplicating  peripherals  is  considered. 

Networking  does  require  that  all  aspects  of  the  instructional  materials  be  in  digital  form.  Audio 
and  video  in  the  digital  form  takes  up  far  more  memory  than  when  in  analog  form.  This  means 
that  the  file  server  must  be  large  enough  to  store  the  set  of  instructional  materials.  Each 
workstation  m.ust  have  enough  memory  to  store  whatever  chunk  of  material  is  down  loaded  by 
the  user.  The  problem  of  full  motion  video  is  still  there  and  may  be  more  acute  than  before 
because  there  is  a  greater  distance  between  the  file  server  and  the  hard  disk  in  the  workstation 
than  between  the  hard  disk  and  the  RAM  in  the  workstation. 

Putting  the  audio  and  video  data  into  digital  form  is  a  fairly  easy  procedure  with  the  right 
hardware  and  software  tools.  Still  pictures  can  be  scanned  into  the  computer  and  audio  recorded 
wi*h  an  audio  editor  and  recording  device  onto  a  hard  disk. 

Networking  does  create  some  problems  for  the  developer  of  the  instructional  materials.  The  set 
of  instructional  materials  must  be  developed  in  pieces  that  will  fit  into  memory  on  the 
workstation.  Extensive  use  of  video  and  audio  must  be  carefully  considered  because  of  memory 
size.  If  a  peripheral  such  as  a  Level  III  videodisk  player  were  attached  directly  to  the 
workstation,  for  example,  it  would  require  no  computer  memory  whatsoever.  Hence,  an 
instructional  designer  must  think  differently  when  a  networked  system  is  involved  as  opposed  to 
a  single  user  system. 

THE  FULL  MOTION  VIDEO  PROBLEM 

With  an  understanding  of  how  networks  can  be  used  with  multimedia  systems,  it  is  appropriate 
to  examine,  in  greater  detail,  the  problem  of  full  motion  video  mentioned  previously  in  the 
paper.  Full  motion  video  is  taken  for  granted  by  learners.  They  have  been  exposed  to  television, 
movies,  and  video  tape  for  mush  of  their  life.  Thus,  they  expect  to  see  it  in  their  learning 
experiences.  Full  motion  video  keeps  their  attention  most  of  the  time. 

The  amount  of  memory  required  to  store  a  full  screen  digital  video  image  is  extremely  large. 
For  example,  a  modestly  detailed  color  image  with  resolution  of  512  by  480  pixels  with  a  16  bit 
color  designation  for  each  pixel  requires  over  491,000  bytes  of  storage.  Even  at  today's  transfer 
rates,  it  still  would  take  sometime  to  move  this  image  from  one  place  to  another. 

Now,  suppose  that  this  image  was  one  in  a  motion  sequence  of  video  images.  A  person  viewing 
the  playback  of  the  motion  sequence  will  not  see  flicker  at  a  playback  rate  of  30  frames  per 
s^'^oad.  In  some  cases,  a  playback  rate  of  20  frames  per  second  will  produce  the  flicker.  In 
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either  case,  the  transfer  rate  technology  today  cannot  support  this  heavy  demand  and  so  some 
alternatives  must  be  used. 

One  technique  that  could  be  used  is  to  store  only  the  part  of  each  image  that  differs  from  the 
previous  image.  This  technique  is  commonly  used  in  computer  graphics  animation.  Since  the 
animation  is  generated  by  a  program,  it  is  relatively  easy  to  create  the  succeeding  image  by 
producing  only  the  part  of  the  image  that  changed.  However,  a  video  image  that  is  scanned  or 
recorded  either  off-line  or  in  real  time  is  not  produced  by  a  program.  Consequently,  the  image 
must  be  changed  by  software  and  the  process  can  become  tedious  and  time  consuming. 

A  second  alternative  is  to  reduce  the  amount  of  screen  area  used  to  display  the  motion  sequence. 
This  reduces  the  number  of  bytes  to  be  transferred.  At  least  one  vendor  is  currently  using  this 
alternative  utilizing  one  quarter  of  the  display  screen. 

Perhaps  the  most  promising  alternative  is  to  use  compression  algorithms  to  reduce  the  amount 
of  storage  required  to  store  an  image.  This  process  can  be  tricky  because  too  much  compression 
may  destroy  desired  picture  detail.  When  the  image  is  to  be  displayed,  it  must  then  be 
decompressed.  As  long  as  the  image  is  a  single  frame  and  not  part  of  a  motion  sequence,  the 
algorithm  can  be  implemented  without  worrying  too  much  about  speed.  However,  spTed  is  a 
main  consideration  when  compressing  and  decompressing  a  motion  sequence  of  images. 
Algorithms  that  have  compression  ratios  of  at  least  200  to  1  are  considered  to  be  minimum. 

Two  adjacent  pixels  that  have  the  same  color  need  store  the  color  information  only  once.  It  is 
not  difficult  to  realize  the  simple  (less  detail)  video  images  are  more  easily  compressed  than 
complex  video  images. 

The  human  vision  system  also  plays  a  role  in  compression  algorithm  development.  For  example, 
there  are  a  number  of  colors  that  are  nearly  indistinguishable  to  the  human  eye;  color  data  for 
pixels  that  use  these  colors  can  be  eliminated,  thus  aiding  the  compression  problem  and 
improving  the  compression  ratio. 

As  yet,  there  have  been  no  standards  developed  for  video  compression  technology.  Progress  is 
being  made  to  this  end  by  the  Instructional  Organization  for  Standards  (ISO).  A  subgroup,  the 
Joint  Photographic  Experts  Group  (JPEG)  is  responsible  for  developing  a  standard  for  still  image 
compression  while  another  subgroup,  the  Moving  Picture  Coding  Experts  Group  (MPEG)  is 
responsible  for,  developing  the  standard  for  motion  image  compression.  The  JPEG  standard  has 
had  initial  approval  but  the  MPEG  standard  is  far  from  completion  because  of  the  complexity 
of  motion  image  compression.  The  DVI  technology  is  as  well  developed  as  any  other  set  of 
compression  algorithms  but  it  is  proprietary  and  hence,  details  of  their  structure  are  not 
available. 

There  is  still  a  great  deal  of  work  to  be  done  on  the  full  motion  video  problem.  In  the  next 
section,  two  solutions,  both  using  compression  algorithms  will  be  discussed.  While  these  solutions 
are  a  first  step,  the  advent  of  digital  television  with  screen  resolution  of  1024  by  1024  will 
continue  to  make  the  motion  sequence  problem  a  sticky  one  for  many  years  to  come. 


604 

W  0  /W 


DVI  AND  QUICK  TIME 

The  most  promising  solutions  to  the  full  screen,  full  motion  video  problem  appears  to  be  the 
digital  video  interactive  (DVI)  technology  under  development  by  Intel  and  IBM  and  the  Quick 
Time  software  by  Apple.  A  brief  ovei-view  of  each  solution  will  be  given  in  this  section. 

Development  of  DVI  began  in  1987  with  the  first  demonstration  of  the  technology  developed  at 
the  Sarnoff  Research  Center.  Its  demonstration,  using  special  boards  in  an  IBM  PC  and  a  CD- 
ROM  disk,  showed  that  72  minutes  of  full  motion  video  could  be  stored  on  the  12  cm  disk.  The 
increased  capability  for  storage  of  full  motion  video  was  made  possible  through  the  use  of  video 
compression  and  decompression  techniques. 

In  1989,  the  DVI  technology  was  sold  to  Intel,  who  is  well  known  for  its  development  of  the  80 
series  (8088,  80286,  80386,  etc.)  CPU  chips.  Intel  is  vigorously  working  to  make  the  DVI 
technology  commercially  feasible  and  is  making  good  progress  to  that  end.  Initially,  the  DVI 
technology  was  being  developed  for  the  IBM  PS/2  and  PC  XT/ AT  compatibles  but  will  likely 
be  extended  to  other  vendors'  products. 

The  DVI  technology  has  four  unique  components: 

a  custom  VLSI  chip  set 

a  specification  for  a  custom  software  interface 

video/audio  data  file  formats 

compression  and  decompression  algorithms 
Hence,  ii  is  clearly  a  combination  of  hardware  and  software  components. 

The  VLSI  chip  set  the  heart  of  DVI  technology.  It  consists  of  two  VLSI  chips  and  acts  as  a 
compressor.  The  speed  of  this  chip  set  is  what  permits  DVI  to  be  able  to  accomplish  its  mission. 

The  functions  of  DVI  are  performed  in  application  programs  written  in  the  C  language.  In  some 
cases,  assembly  language  is  used.  For  users  who  are  not  C  programmers,  authoring  systems  are 
available.  Specifications  are  given  to  the  authoring  system  which  generates  the  necessary  DVI 
code. 

The  development  of  digitized  audio  and  video  has  had  and  will  continue  to  have  a  significant 
impact  on  LANs.  It  means  that  full  multimedia  capabilities  can  be  shared  by  multiple 
workstations  on  a  LAN  and  with  TCP/IP,  with  multiple  LANs. 

There  are  a  number  of  applications  where  the  DVI  and  LAN  technologies  can  work  together. 
One  is  video  conferencing.  In  this  case,  a  meeting  can  be  held  with  group  members  at  remote 
sites.  Participants  can  view  and  hear  each  other  in  real  time  and  rraterials  can  be  shared  with 
ojther  participants  in  real  time.  Time  wasted  in  moving  from  an  office  to  a  control  meeting  site 
is  eUminated. 

A  second  set  of  appUgations  for  DVI  and  LANs  is  the  area  of  education  and  training.  Currently, 
to  achieve  full  multimedia  capability,  a  workstation  for  each  individual  must  be  equipped  with 
additional  peripherals  such  as  a  CD-ROM  drive.  Such  workstations  have  become  known  as 
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digital  video  interactive  (DVI)  technology  under  development  by  Intel  and  IBM  and  the  Quick 
Time  software  by  Apple.  A  brief  overview  of  each  solution  will  be  given  in  this  section. 

Development  of  DVI  began  in  1987  with  the  first  demonstration  of  the  technology  developed  at 
the  Sarnoff  Research  Center.  Its  demonstration,  using  special  boards  in  an  IBM  PC  and  a  CD- 
ROM  disk,  showed  that  72  minutes  of  full  motion  video  could  be  stored  on  the  12  cm  disk.  The 
increased  capability  for  storage  of  full  motion  video  was  made  possible  through  the  use  of  video 
compression  and  decompression  techniques. 

In  1989,  the  DVI  technology  was  sold  to  Intel,  who  is  well  known  for  its  development  of  the  80 
series  (8088,  80286,  80386,  etc.)  CPU  chips.  Intel  is  vigorously  working  to  make  the  DVI 
technology  commercially  feasible  and  is  making  good  progress  to  that  end.  Initially,  the  DVI 
technology  was  being  developed  for  the  IBM  PS/2  and  PC  XT/AT  compatibles  but  will  likely 
be  extended  to  other  vendors'  products. 

The  DVI  technology  has  four  unique  components: 

a  custom  VLSI  chip  set 

a  specification  for  a  custom  software  interface 

video/audio  data  file  formats 

compression  and  decompression  algorithms 
Hence,  it  is  clearly  a  combination  of  hardware  and  software  components. 

The  VLSI  chip  set  the  heart  of  DVI  technology.  It  consists  of  two  VLSI  chips  and  acts  as  a 
compressor.  The  speed  of  this  chip  set  is  what  permits  DVI  to  be  able  to  accomplish  its  mission. 

The  functions  of  DVI  are  performed  in  application  programs  written  in  the  C  language.  In  some 
cases,  assembly  language  is  used.  For  users  who  are  not  C  programmers,  authoring  systems  are 
available.  Specifications  are  given  to  the  authoring  system  which  generates  the  necessary  DVI 
code. 

The  development  of  digitized  audio  and  video  has  had  and  will  continue  to  have  a  significant 
impact  on  LANs.  It  means  that  full  multimedia  capabilities  can  bet  shared  by  multiple 
workstations  on  a  LAN  and  with  TCP/IP,  with  multiple  LANs. 

There  are  a  mimber  of  applications  where  the  DVI  and  LAN  technologies  can  work  together. 
One  is  video  conferencing.  In  this  case,  a  meeting  can  be  held  with  group  members  at  remote 
sites.  Participants  can  view  and  hear  each  other  in  real  time  and  materials  can  be  shared  with 
other  participants  in  real  time.  Time  wasted  in  moving  from  an  office  to  a  control  meeting  site 
is  eliminated. 

A  second  set  of  applications  for  DVI  and  LANs  is  the  area  of  education  and  training.  Currently, 
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606 


multimedia  personal  computers  (MFC).  With  DVI,  one  MFC  can  be  designated  as  a  server  to 
a  LAN  with  other  workstations  that  do  not  have  the  peripherals.  These  remote  workstations  can 
share  the  content  of  one  or  more  CD-ROM  using  the  LAN,  thus  reducing  the  cost  of  each 
workstation. 

Unlike  the  DVI  technology  which  uses  both  hardware  and  software,  Quick  Time,  developed  by 
Apple,  uses  software  exclusively  to  do  its  compression/decompression  work.  Both  audio  and 
video  can  be  compressed  and  put  directly  into  an  applications  program  (stack)  developed  by  the 
use  of  HyperCard,  Super  Card,  or  Authorware  among  others. 

Full  motion  video  can  be  put  into  the  application  either  off-line  (such  as  a  movie  or  videotape) 
or  in  real  time  using  a  video  camera.  The  video  and  accompanying  audio  can  then  be  edited, 
if  necessary,  to  fit  the  needs  of  the  application.  Currently,  Quick  Time  will  permit  only  a  one 
eighth  screen  size  window  on  the  display  but  this  capability  is  scheduled  to  expand  to  one  quarter 
screen  in  October  of  1992.  No  prediction  is  yet  made  by  Apple  relative  to  full  screen,  full 
motion  video. 

Introduced  in  January  1992,  Quick  Time  has  the  advantage  that  it  needs  no  additional  hardware, 
except  memory  like  DVI,  it  will  operate  with  a  local  area  network. 

SUMMARY 

Multimedia  systems  will  play  an  integral  role  in  education  and  training,  both  in  the  present  and 
in  the  future.  The  major  abstract  to  a  complete  multimedia  system  that  is  completely  digital  is 
full  motion  video  on  a  full  screen  display.  Solutions  to  this  problem  have  been  developed  by  the 
use  of  video  compression/decompression  algorithms  that  have  produced  less  than  full  screen,  full 
motion  video.  It  is  anticipated  that  with  JPEG  and  MPEG  standards,  that  full  screen,  full 
motion  video  is  close  at  hand. 
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Abstract 

This  paper  examines  educational  applications  of  computer-based  communications  to  identify  unique 
tools  for  learning  provided  by  this  medium  and  to  identify  key  features  of  successful  computer 
mediated  communications  (CMC)  projects.  A  review  of  CMC  programs  provides  guidelines  for 
designing  effective  teacher  training  programs  in  CMC.  Since  institutions  of  higher  learning  are 
significant  sponsors  of  technology  training  programs  for  teachers,  an  additional  section  addresses  the 
role  of  higher  education  in  supporting  improvements  in  teacher  training  programs. 

Introduction 

In  1984  Kiesler,  Siegel,  and  McGuire  observed  that "...  no  one  can  predict  in  any  detail  the  nature  of 
the  transformations  that  computers  will  bring,  but  one  aspect  of  Hfe  that  will  certainly  be  affected  is 
communication"  (p.  1123).  Today  thousands  of  school  children,  adult  professionals  and  hobbyists 
use  computers  to  send  millions  of  electronic  messages,  files,  and  data  to  colleagues  on  a  daily  basis. 
In  fact,  computer-based  network  communication  is  increasing  at  the  astonishing  rate  of  twenty-five 
percent  per  month-an  increase  which  would  put  every  single  human  being  online  within  a  few 
decades  (Barlow,  1992,  p.  25).  Nonetheless  relatively  little  is  known  about  the  social  and 
psychological  impact  of  computer-mediated  communications  on  those  who  communicate. 

The  technical  aspects  of  computer-mediated  communication  (CMC)  have  made  this  medium  an 
attractive  choice  for  widely  var>'ing  purposes  including  instructional  programs  and  courses  (Kaye, 
1989).    CMC  can  provide  interactive  capabilities  which  are  not  bound  by  time  and  geographic 
distance,  and,  compared  to  the  telephone  and  mail  service,  electronic  communications  offer  increased 
flexibility  and  speed  at  a  significantiy  reduced  cost.  Proponents  of  computer  communications  claim 
that  the  greatest  merits  of  well  structured  network  projects  in  education  are  that  they  help  students 
learn  specific  content  knowledge  as  well  as  certain  academic  skills-especially  language  skills  (Riel, 
1991-1992,  p.  5).  However,  such  relatively  uncritical  praise  may  not  recognize  the  actual  impact  of 
CMC  -  both  positive  and  negative. 

In  this  article  we  review  CMC  from  the  perspective  of  Richard  Clark's  premise  (1983)  that  media  in 
and  of  themselves  do  not  affect  learning.  Instead  we  assume  that  particular  unique  qualities  of  any 
medium  including  CMC  interact  with  learners*  aptitudes  and  preferences.  Our  goal  is  to  identify  the 
intrinsic  characteristics  of  CMC  and  analyze  how  Uiese  media  traits  tend  to  interact  with  the  learner, 
situation  of  learning,  and  mode  of  instruction.  This  may  provide  a  framework  to  determine 
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educational  contexts  and  objectives  for  which  computer-based  interaction  may  be  particularly  well 
suited. 

Background 

Clark  and  Sugruc  (1991)  suggest  four  key  dependent  variables  as  being  most  frequently  studied  by 
media  researchers:  (l)performance  outcomes;  (2)cognitive  processing  (type  of  learning  being 
measured);  (3)efficiency/costs;  and  (4)  equity  of  access  to  instruction.  While  acknowledging  that 
their  list  of  independent  variables  may  be  expanded  by  others,  CJark  and  Sugrue's  three  main 
variables  are:  (l)media  characteristics  (including  type  of  medium,  specific  attributes  of  medium, 
symbol  systems  within  medium);  (2)student  characteristics  such  as  general  ability,  attributions, 
preferences  and  prior  knowledge;  and  (3)instructional  method  such  as  on-the-job  training,  lecture, 
individualized/group  instruction,  and  programmed  instruction  (Goldstein,  1985). 

We  believe  that  CMC  research  requires  some  additional  conditions  for  analysis  because  of  the 
relation  between  communication  context  and  meaning,  an  issue  not  accounted  for  in  Clark  and 
Sugrue's  variables.  While  the  technical  capabilities  of  CMC  are  well  documented  the  social 
psychological  aspects  of  computer-based  communication  are  not  yet  clearly  understood  (Feenberg, 
1987;  Kiesler  et  al.,  1984;  McGuire,  Kiesler,  &  Siegel,  1987;  Siegel,  Dubrovsky,  Kiesler,  & 
McGuire,  1986).  Much  of  the  enthusiasm  stimulating  instructional  use  of  CMC  is  based  on  "a  shift 
away  from  stand-alone  computer  and  individual  software  packages  to  technology  that  is  more 
focused  on  connecting  students  and  teachers  to  each  other  and  to  the  world  outside  the  school  walls" 
(November,  1992,  p.  50).  In  a  commentary  discussing  computer-to-computer  conversations,  Janet 
Perrolle  (1991)  suggests  that  CMC  should  be  viewed  as  a  social  rather  than  a  psychological  or 
cognitive  phenomenon  and,  as  such,  use  of  CMC  should  be  carefully  planned  to  establish 
"nondistorted  communication"  (p.  23).  Therefore,  we  have  added  to  the  Clark  and  Sugrue  framework 
for  media  research  two  variables  which  address  the  complexity  of  the  social  situation  in  which  the 
learning  occurs.  The  social  context  of  learning  and  instruction  will  be  considered  an  independent 
variable  and  the  nature  of  social  interaction  (descriptive  information  regarding  the  kinds,  distribution, 
fivjquency,  and  content  analysis  of  social  interactions)  will  be  considered  a  dependent  measure  used 
for  analysis  (See  Table  1). 

As  an  initiating  statement,  we  have  found  that  the  literature  on  CMC  ranges  from  enthusiastic 
testimonials  to  quasi-experimental  studies  to  a  few  controlled  experiments.  The  testimonials  by 
classroom  teachers  and  education  researchers  are  helpful  in  motivating  others  to  consider  potential 
uses  for  this  new  instructional  medium.  However,  it  is  possible  that  CMC  enthusiasts  who  have  not 
considered  each  of  the  media  research  variables  listed  in  Table  1  may  unknowingly  be  basing  their 
claims  of  success  with  CMC  on  uncontrolled  and  novelty  effects.  Quasi-experimental  comparisons 
of  CMC  with  existing  instruction  also  base  some  of  their  successful  outcomes  on  uncontrolled  factors 
such  as  instructional  method  and  student/teacher  attitudes.  Nonetheless  these  "real-world," 
situationally-based  application  studies  (e.g.,  Beals,  1991;  Feenberg,  1987;  Kaye,  1989;  Quinn,  Mehan, 
Levin,  and  Black,  1983;  Romiszowski,  1990)  provide  valuable  information  about  CMC  technology, 
its  uses,  and  its  potential  strengths  and  weaknesses. 
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Scientific  experiments  (e.g.,  Kiesler  et  al.,  1984;  Siegel  et  al.,  1986;  McGuire  et  ah,  1987)  provide  a 
more  solid  foundation  of  knowledge  upon  which  educators  can  begin  to  construct  an  understanding 
of  how  CMC  can  be  effectively  used  as  a  medium  for  instnicdon.  However,  it  is  clear  that 
educational  application  of  computer  mediated  interaction  is  not  easily  reduced  to  scientifically 
controlled  conditions.  The  primary  challenge  with  social  context  variables  such  as  those  associated 
with  computer-based  interactions,  is  to  establish  control  in  order  to  make  comparison  with  similarly 
organized  face-to-face  situations.  Because  electronic  dialogue,  at  this  time,  is  primarily  text-based 
interchange,  it  is  an  ideal  medium  f^:  conducting  social  interaction  research  of  this  type  as  the 
context  and  structure  can  be  controlled  by  the  discussion  moderator  and  all  interactions  are  archived 
in  electronic  text  form. 


Table  1 

Dependent/Independent  Variables  Associated  with  Media  Research 


Dependent  Variables 

1)  Performance  outcomes 

2)  Cognitive  processing  -  type  of  learning 
being  measured 

3)  Efficiency/costs 

4)  Equity  of  access  to  instruction 

5)  Nature  of  social  interaction  - 

kind,  distribution,  frequency,  and  content 
analyses. 


Independent  Variables 

1)  Media  characteristics  -  type  of  medium, 
specific  attributes  of  medium,  and  symbol 
systems  within  medium 

2)  Student  characteristics  -  including  general 
ability,  attributions,  preferences,  and  prior 
knowledge 

3)  Instructional  method  such  as  on-the-job 
training,  lecture,  individualized  or  group 
instruction,  programmed  instruction 

4)  Social  context  of  learning  and  :rstmction 


Independent  VarabJes 

Unique  characteristics  of  this  medium.  The  term  computer-mediated  communication  refers  to 
electronic  mail  and  messages,  electronic  bulletin  board  systems,  computer  conferencing  systems,  and 
other  forms  of  computer-based  network  communications.  Here  we  address  only  text-based  CMC,  but 
state-of-the-art  CMC  technology  can  include  interactive  audio,  video,  and  graphics  communications 
Because  CMC  is  an  emerging  technology,  research  regarding  the  influence  of  the  characteristics  oi 
this  medium  and  their  impact  will  change  as  new  features  and  capabilities  are  incorporated  into 
standard  CMC  technology. 

CMC  offers  previously  unimaginable  communications  access  and  capabilities.  Computer 
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communications  technology  enables  student  and  teachers  to  exchange,  store,  edit,  broadcast,  copy, 
and  send  written  documents  instantly,  conveniently,  and  relatively  cheaply  over  short  or  long 
distances  (Kiesler  et  al,  1984).  Interactions  via  computer  exchanges  may  be  immediate  or  delayed 
regardless  of  the  geographic  distance  between  participants.  For  example,  with  the  use  of  network 
technology  two  or  more  individuals  can  view  the  same  document  and  make  simultaneous  revisions 
without  meeting  f«ce-to-face  or  using  a  telephone.  Anthony  Kaye  suggests  that  CMC  "...is 
essentially  a  medium  of  written  discourse,  which  shares  some  of  the  spontaneity  and  flexibility  of 
spoken  conversation..."  (1990,  p.  10).  Others  a>avie  &  Wells,  199.1;  Harasim,  1990)  point  out  that 
CMC  is  capable  of  the  speed  and  interactivity  of  telephone  and  face-to-face  communication,  but 
lacks  the  aural  and  visual  cues  associated  with  telephone  and  person-to-person  exchange. 

From  a  physical  perspective,  computer-to-computer  exchanges  are  based  on  activities  which  are  very 
familiar  such  as  kcyboarding  and  printing  messages,  using  standard  syntax.  We  have  been 
"speaking"  to  others  via  text  messages  in  traditional  print  media  using  such  technologies  for 
centimes.  It  appears  that  computer-mediation  changes  the  nature  of  these  communications  in  two 
ways. 

First,  the  asynchronous  nature  of  computer  interactions  appears  to  cause  the  pattern  of  discussion  in 
CMC  to  differ  from  face-to-face  exchanges.  CMC  responses  can  be  immediate  or  delayed  and  may 
be  to  one  or  to  many  individuals  who  may  be  geographically  close  or  on  the  opposite  side  of  the 
globe.  Face-to-face  exchanges  tend  tc  be  linear  so  that  remarks  and  replies  build  from  one  another. 
For  example,  in  a  class  situation  matters  of  procedure  tend  to  be  clustered  at  the  beginning  and  end 
of  classroom  lessons  (Quinn  et  al.,  1983).  In  contrast,  online  discussions  are  characterized  as  having 
"multiple  threads"  to  the  discussion  and  a  non-linear  pattern.  Procedural  questions  often  continue 
throughout  the  entire  exchange,  running  parallel  to  on-going  discussion  of  several  other  topics  (Kaye, 
1989;  Kiesler  et  al.,  1984;  Quinn  et  al.,  1983).  This  aspect  of  CMC  can  be  influenced  to  some 
extent  by  the  moderator  of  the  discussion  who  serves  a  purpose  similar  to  a  moderator  of  face-to-face 
meetings  by  keeping  the  group  focussed  and  on  task  (Feenberg,  1987),  However,  CMC  clearly 
makes  possible  communication  patterns  not  found  in  face-to-face  interactions. 

A  second  unique  aspect  of  CMC  is  that  users  ot  this  technology  can  create  their  own  syntax  and 
lexical  symbol  system  to  convey  and  clarify  affective  expression;  CMC  users  also  can  control  the 
timing  of  communications  interactions.  Because  CMC  is  primarily  limited  to  text-based  interaction 
and  involves  a  limited  social  context,  users  generate  electronic  analogs  of  common  communication 
features  such  as  body  language  and  voice  inflection.  Deals'  analysis  of  first-year  teachers' 
interactions  on  Harvard''?  Beginning  Teachers  Network  reports  that  participants  embellished  their 
informal  exchanges  with  "textual  displays  of  what  would  have  been  non-verbal  cues  in  face-to-face 
conversation"  (1991,  p.  76).  Other  CMC  researchers  (Kiesler  et  al,,  1984)  also  report  use  of 
exaggerated  punctuation  and  lexical  symbols  in  electronic  mail  and  computer  conference  exchanges 
to  convey  tone  and  gesture  cues  to  the  reader.  It  appears  that  these  contextual  aspects  of 
communication  are  sufficiently  important  to  establishing  meaning  that  creative  adaptations  have 
generated  symbols  to  reduce  the  contextual  sterility  of  CMC.  As  for  timing,  CMC  users  can  respond 
on  a  personal  time  schedule.  As  a  result,  it  appears  that  CMC  communications  can  be  subjected  to 
different  types  of  consideration  from  the  sender  and  can  evoke  different  types  of  responses  than  face- 
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to-face. 

Student  characteristics.  Margaret  Riel  (1991-1992)  points  out  that  failures  to  create  effective 
electronic  projects  by  and  large  go  unreported.  It  may  be  that  mosc  of  the  reports  on  network 
projects  are  biased  by  the  self-selection  of  their  creators  and  participants.  The  classroom  study 
conducted  by  Quinn  et  al.  (1983)  found  that  anxiety  about  using  a  computer  was  a  significant  factor 
to  consider  when  introducing  a  new  computer-based  medium  to  adults.  It  appears  that  those  who 
have  some  fear  or  negative  feelings  towards  using  computers  self-select  themselves  out  of  CMC 
projects  whenever  possible.  The  same  report  by  Quinn,  et  al.  (1983)  found  that  computer  anxiety 
was  not  factor  when  introducing  computer  technology  to  elementary-age  children,  and  computer 
anxiety  may  be  much  less  common  today  than  it  was  ten  years  ago. 

In  her  report  on  participants'  reactions  to  the  delay  in  response  time  to  online  interactions,  Beals 
(1991)  found  that  some  participants  felt  that  the  time  lag  allowed  more  time  to  think  and  to  feel  safer 
before  responding.  Other  participants  reported  that  the  delay  in  receiving  responses  made  them  feel 
"less  engaged"  in  the  discussion.  Investigations  of  the  level  of  involvement  and  attentiveness  of 
participants  in  face-to-face  versus  electronic  message  systems  verses  computer  conference  system 
among  familiar  and  anonymous  exchanges  indicate  that  individuals  are  less  psychologically  aroused 
when  viewing  anonymous  postings  (Kiesler  et  al.,  1984). 

Unanswered  questions  remain  regarding  how  CMC  exchanges  are  influenced  by  the  tendency  of  the 
medium  to  redirect  the  users  attention  from  their  audience  and  under  what  circumstances  and  for 
what  individuals  CMC  may  tend  to  stimulate  feelings  of  dehumanization  and  detachment.  The 
documented  increase  in  uninhibited  verbal  behavior  such  as  "flaming"  in  computer-based  conferences 
also  reflects  a  decreased  awareness  of  audience  in  electronic  interactions.  The  above  conditions  were 
found  in  text-only  communication.  More  sophisticated  technologies  may  provide  more  social  cues 
by  creating  conditions  more  similar  to  face-to-face  exchanges.  Nonetheless,  it  appears  that  CMC 
does  influence  the  pace,  duration,  and  content  of  communication  as  well  as  the  affective  quality  of 
messages. 

Instructional  method.  Clark  and  Sugrue  (1991;  Clark,  1983)  suggest  that  conclusions  from  media 
research  are  frequently  confounded  by  two  factors:  (l)uncon trolled  effects  caused  by  variations  in 
instructional  method  or  content  differences  and  (2)novelty  effects  associated  with  newer  media 
which  tend  to  disappear  over  time.  Particular  attention  must  be  given  to  these  factors  in  any 
comparison  of  CMC  with  traditional  instructional  media  because  online  learning  is  a  currently 
popular  medium  and  is  receiving  a  lot  of  attention  at  this  time. 

Whether  the  instructional  method  used  is  situationally-based  (such  as  on-the-job  training), 
individualized  or  mass  produced  programmed  instruction,  CMC  is  a  viable  choice  for  instructional 
delivery.  Pilot  research  by  Britain's  Free  University  indicates  that  use  of  CMC  provides  a  means  for 
enriching  home-bound  students'  interaction  with  their  tutor  and  fellow  students.  Instructional  media 
selection  relies  on  an  accurate  assessment  of  the  need  for  visual,  audio,  sensory,  hands-on  practice, 
demonstration  and  discussion/interaction.  Kaye  (1989)  suggests  that  the  additional  costs  associated 
with  establishing  and  managing  CMC  versus  traditional  distant  learning  media  should  be  compared 
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with  the  benefits  of  increased  interaction  with  instructor  and  fellow  students.  Frequently  educational 
applications  of  new  technology  stimulate  creative  instructional  methods  for  delivering  information. 
The  new  instructional  methods  such  as  greater  student-to-student  interaction  is  not  medium- 
dependent,  but  may  reflect  capabilities  of  a  given  medium  which  make  alternative  teaching  styles 
more  apparent  and  more  manageable  (Feenberg,  1987). 

Definition  of  social  context.  Just  like  the  traditional  classroom,  a  particular  electronic  community 
has  its  own  social  milieu.  The  goals  and  objectives  which  inaugurated  a  group  also  play  a 
continuing  role  in  affecting  the  social  structure  of  the  group  as  does  each  individual  participant's 
attitudes  towards  the  group. 

In  Tombaugh's  (1984)  study  of  a  technical  expert  discussion  group,  the  more  highly  regarded 
scientists  enrolled  in  the  conference  group  reported  that  they  were  too  busy  to  participate  in  the  on- 
going group  interactions  and  were  less  motivated  to  respond  to  the  continuing  electronic  dialogue. 
Other  (less  individually  prominent)  participants  tended  to  direct  their  questions  and  comments  to  the 
more  prestigious  participants.  The  failure  of  this  social  interaction  has  many  possible  explanations; 
but,  it  is  obvious  that  the  prominent  scientists  did  not  share  a  common  purpose  with  their  lesser 
known  colleagues.  In  addition,  many  of  the  scientists  were  reluctant  to  share  unpublished  research 
over  the  network.  This  mistrust  among  group  members  reported  by  Tombaugh  is  not  a  medium 
dependent  problem;  but,  it  does  reflect  the  social  problems  resulting  from  issues  not  clearly  discussed 
or  resolved  among  members  of  any  group. 

Thus,  it  may  be  that  context  in  CMC  is  the  product  of  stated  agreements  or  time-based  informal 
negotiations  just  as  in  face-to-face  communication.  The  absence  of  context  cues,  however,  may 
increase  the  importance  of  context  agreements  for  successful  communication  that  includes 
considerable  interaction  and/or  revelation  of  personal  or  proprietary  information. 

Other  complexities  associated  with  face-to-face  group  activities  that  also  occur  in  CMC  interactions 
arc  evidenced  in  different  ways.  Because  computer-based  interaction  is  capable  of  a  variety  of  time 
and  space  configurations  for  interactions,  the  structure  of  CMC  discussions  is  more  complex  than 
face-to-face  exchange.  Computer  discussions  typically  exhibit  multi-level  topical  themes  being 
discussed  simultaneously  at  varying  rates  of  response  by  participants  (Romiszowski  &  de  Hass, 
1989).  This  suggests  that  the  social  context  in  which  CMC  discourse  occurs  exhibits  a  unique 
structure  and  characteristics  of  interactions  which  may  be  physically  impossible  to  replicate  via  other 
communication  media. 

Dependent  variables 

Performance  outcomes(type  of  testing/assessment  of  learning).  Several  applications  of  CMC 
projects  measure  student  performance  outcome  variables  to  determine  whether  students  who  have 
learned  content  material  via  CMC  show  similar  performance  outcomes  as  their  peers  who  have  been 
through  face-to-face  interaction.  Quinn  et  al.  (1983)  found  no  significant  differences  in  CMC  versus 
student  face-to-face  scores  on  tests  of  course  content,  but  the  nature  of  student-to-student,  teacher-to- 
student,  and  studcnt-to-student  interactions  differed.  Thus,  in  this  case,  different  instructional  media 
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with  different  patterns  and  content  of  interactions  yielded  basically  similar  performance  outcomes.  In 
contrcit,  Kiesler  et  aL  (1984)  and  Siege!  (1986)  used  previously  tested  group  decision-making  tasks 
to  compare  CMC  versus  face-to-facc  interaction.  These  studies  of  CMC  showed  that  the  outcome  of 
group  decision-making  was  significantly  influenced  by  the  type  of  media  available  for  group 
communication.  The  CMC  groups  tended  to  make  less  predictable  decisions  than  the  face-to-face 
groups. 

The  results  of  these  studies  appear  to  be  inconsistent  and  suggest  that  farther  research  is  needed  to 
determine  more  about  hov/  learning  via  CMC  affects  learning  outcomes.  However,  it  may  be  that 
differences  in  outcomes  are  dependent  on  the  nature  of  the  task;  the  research  above  suggests  that 
higher  order  learning  skills  such  as  decision-making  and  problem  solving  are  more  affected  by  the 
social  context  associated  with  the  instructional  delivery  method. 

Cognitive  processing  {type  of  learning).  Riel  (1991-1992),  Davie  and  Wells  (1991),  and  Goldman 
and  Newman  (in  Roberts,Blakeslee,  Brown,  &  Lenk,  1990,  p.  99)  described  the  majority  of  CMC 
projects  as  having  markedly  different  objectives  from  traditional  curriculum  programs.  They  point 
out  that  CMC  projects  typically  require  cross-curricular  skills  in  problem  solving  tasks.  Goldman 
and  Newman  suggest  that  CMC  is  most  effective  as  a  tool  to  enrich  the  classroom  learning 
experience  in  the  following  five  situations:  (l)identified  topics  and  group  conferences;  (2)non- 
competitive  activities;  (3)questions  that  require  more  reflection;  (4)quesuons  where  answers  are 
wanted  from  several  students;  and  (5)private  interactions.  In  addition,  Goldman  and  Newman 
propose  that  electronic  communication  is  not  an  appropriate  tool  for  teaching  new  concepts  that 
require  intense,  fast-paced  interactions. 

Efficiency/costs.  Educational  applications  of  CMC  have  been  found  to  be  fast,  relatively 
inexpensive,  and  convenient  (Heals,  1991).  The  true  test  of  the  cost  efficiency  of  using  CMC  is 
addressed  in  Riel's  (1991-1992)  question  for  designers  of  network  projects  to  answer:  Is  the  time 
invested  in  network  activities  matched  by  significant  return  in  student  learning  or  teacher 
development?  To  some  extent  individual  teachers  and  school  systems  will  have  to  answer  this 
question  based  on  their  student  needs  and  budgetary  restrictions.  Attempts  to  answer  this  question 
will  be  highly  speculative  until  a  sufficient  database  is  developed  from  which  careful  comparison  can 
be  made. 

Equality  of  access  to  instruction*  From  a  purely  technical  perspective,  CMC  is  attractive  to  many 
because  it  can  be  an  extremely  rapid  and  efficient  way  of  transmitting  information.  However,  the 
greater  convenience  afforded  by  CMC  comes  only  after  all  the  technical  skills  needed  to  use  CMC 
have  been  mastered.  CMC  research  available  at  this  time  rarely  considers  the  current  technical 
requirements  to  establish  an  efficient  network  system  described  by  one  network  expert  as  being  "not 
for  the  technically  timid"  (W.  Sanders,  personal  communications,  September  xx,  1991).  Frustrations 
due  to  technical  naivete  and  the  lack  of  support  personnel  have  resulted  in  some  new  users  never 
achieving  the  necessary  level  of  proficiency  in  their  system  or  personal  skills  to  surmount  technical 
difficulties. 

Some  users  of  CMC  are  not  counted  in  CMC  research  because  they  have  been  eliminated  by  self- 
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selective  withdrawal  (Rial,  1991-1992),   For  others,  the  problem  is  more  lack  of  convenient  or  any 
access  to  the  necessary  hardware  to  build  a  network  workstation  (Tombaugh,  1984).  Thus  we  find 
both  persona!  and  technical  issues  that  have  an  impact  on  access  and  the  influence  of  both  issues  is 
not  clear. 

Description  of  social  interactions.  Several  variables  that  appear  consistently  throughout  the 
research  on  social  aspects  of  CMC  are:  degree  of  equality  of  participation,  degree  of  uninhibited 
verbal  behavior,  length  of  response,  and  pattern  of  communication  among  group  members.  Generally 
CMC  interactions  are  more  equally  distributed  among  group  members,  but  the  structure  of  the  group 
and  active  presence  of  a  group  moderator  can  affect  patterns  of  communication  between  group 
members  (Kiesler  et  al.,  1984;  Siegel,  et  al.,  1986;  Quinn  et  al.,  1983;  Tombaugh,  1984).  It  must 
also  be  remembered  that  CMC  typically  includes  a  very  select  group  due  to  their  skill  and  hardware 
access.  Nonetheless,  individuals  exhibit  .lore  uninhibited  verbal  behavior  in  CMC  exchanges  than  in 
face-to-face  interaction  (Kiesler  et  al.,  1984;  Siegel  et  al.,  1986).  And,  responses  via  electronic 
communication  tend  to  be  longer  than  in  face-to-face  communication.  Quinn  et  aL  (1983)  found  that 
the  normal  pattern  of  classroom  question  and  response  was  different  in  CMC  exchanges,  in  that  more 
student-to-student  interaction  occunied,  as  well  as  less  frequent  overt  evaluation  of  student  comments 
by  the  instructor. 

In  their  study  of  group  processes  and  CMC  Siegel,  Dubrovsky,  Kiesler,  and  McGuire  (1986)  reported 
greater  equality  of  participation  and  greater  shift  in  choice  in  the  computer-based  discussion  than  in 
face-to-face  interaction.  These  researchers  also  found  that  CMC  group  decisions  deviated  more  from 
initial  individual  preferences  than  the  face-to-face  group.  This  finding  lead  these  investigators  to 
speculate  that  some  people  will  respond  differently  in  different  communication  settings.  The  lack  of 
nonverbal  or  auditory  cues  available  in  current  electronic  communications  systems  seem  to  result  in 
participants  feeling  more  detached  from  the  communications. 

Tombaugh's  study  of  an  international  scientific  computer-based  conference  did  not  exhibit  the 
equality  of  participation  evidenced  in  nearly  every  other  CMC  study.  Participants  felt  that  the  lack 
of  structure  and  need  for  a  group  moderator  would  have  comccted  this  situation.  Tombaugh  also 
reported  that  the  CMC  conference  tended  to  have  many  more  readers  than  contributors.  This  is  true 
of  many  electronic  communications;  that  is  many  more  messages  arft  sent  than  are  received  (Huff, 
Sproull,  and  Kiesler,  1989).  It  does  appear  that  there  is  sufficient  evidence  from  research  and  theory 
to  suspect  that  CMC  will  indeed  influence  the  nature  of  social  interactions.  The  degree,  direction, 
and  importance  of  these  changes  could  be  a  significant  factor  to  consider  in  assessing  uses  of  CMC. 

Future  Research  Agenda 

It  is  important  to  note  that  the  current  literature  on  CMC  is  far  from  rich.  The  vast  majority  of 
writing  is  descriptive  and  testimonial  in  nature.  This  is  augmented  by  a  group  of  evaluation  studies 
that  primarily  repwrt  instructor  observations  and  participant  reports  regaiding  CMC  projects.  A  very 
small  portion  of  this  literature  approaches  the  standards  required  for  conceptually  careful  and 
controlled  investigations  of  CMC  variables.  Consequently,  conclusions  about  CMC  must  be  weighed 
carefully  in  regard  to  claims  advanced  and  research  support. 
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The  current  CMC  literature  points  toward  the  need  to  investigate  additional  factors  under  the  broad 
heading  of  on-going  media  research  with  online  learning  and  electronic  communities.  A  recent 
electronic  report  by  Cole,  Beam,  Kam,  &  Hoad-Reddick  (1992)  provides  a  brief  summaiy  of  current 
CMC  research  including:  Using  a  concept  mapping  tool  to  assess  problem  solving  strategies  among 
small  network  groups  (Heeren,  E.);  Identifying  relationships  between  communication  media  and 
personality  type  (Carroll,  R.,  &  R.  Schipke);  Use  of  CMC  to  facilitate  seminar  participation  and 
active  thinking  (Shedletsky,  L.);  Exploration  of  ways  to  conduct  online  "focus  group"  interviews 
(Robertson,  J.);  Use  of  online  communication  as  an  enrichment  resource  for  adolescents  with 
disabilities  (Lang,  S.);  and  Assessing  the  effectiveness  of  CMC  to  provide  scientific  r^^le  models  to 
under  represented  groups  (Murfm,B.)-  Each  includes  the  interactive,  social  aspects  of  CMC  as 
dependent  measures  being  analyzed  in  the  investigations.  Consequently,  we  should  soon  begin  to 
amass  more  data  to  evaluate  critically  the  most  appropriate  uses  and  clientele  for  CMC. 

One  of  the  most  exciting  outcomes  of  CMC  rese'-xh  is  finding  that  this  new  medium  has  provided 
new  insight  into  the  complexity  and  power  of  live  face-to-face  ir  teraction.  In  addition,  research  has 
demonstrated  the  potential  for  valuable  local  and  geographically  distant  interaction  via  CMC. 
Obviously,  as  Christopher  O'Malley  (1991)  prophesies,  there  will  not  be  a  single,  best  instructional 
use  of  CMC  or  any  other  technology.  Existing  technologies  are  not  made  obsolete  by  new  media, 
but  all  modes  of  instruction  should  be  updated  as  needs,  facilities,  and  student  mterests  change  and 
new  knowledge  emerges.  However,  it  is  important  for  research  to  expand  and  become  more 
sophisticated.  The  limited  number  of  studies,  restricted  as  they  are  to  select  subjects  and  settings, 
provide  a  relatively  modest  database  from  which  only  the  most  tentative  conclusions  may  be  drawn. 

We  believe  that  CMC  should  be  viewed  as  an  opportunity  for  new  development  and  understanding. 
As  Herbert  Simon  (1987)  so  eloquently  said,  "We  have  to  think  about  technology  in  terms  of  human 
knowledge... new  technology  is  simply  new  knowledge;  and  as  such,  it  resides  not  in  machines  but  in 
the  human  brains  that  invent  them,  develop  them,  and  use  them."  Thus,  by  putting  CMC  to  use  in 
various  ways  and  carefully  considering  the  characteristics  of  the  medium,  the  aptitudes  and 
preferences  of  the  students  involved,  the  appropriate  instructional  method,  and  the  social  context  in 
which  the  instruction  and  learning  are  to  occur,  we  shall  expand  our  knowledge  about  learning  and 
consequendy  about  ourselves  as  well. 
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